molyso.generic package

Generic module. Collection of various helper functionality, from library routines for signal processing, to os specific workarounds.

molyso.generic.etc module

etc.py contains various helper functions and classes, which are not directly related to data processing.

class molyso.generic.etc.BaseCache(filename_to_be_hashed, ignore_cache='nothing', cache_token=None)

Bases: object

A caching class

contains(key)

Parameters:key – Returns:

static deserialize(data)

Parameters:data – Returns:

get(key)

Parameters:key – Returns:

static prepare_key(key)

Parameters:key – Returns:

static serialize(data)

Parameters:data – Returns:

set(key, value)

Parameters:

• key –
• value –

Returns:

molyso.generic.etc.Cache

alias of molyso.generic.etc.FileCache

class molyso.generic.etc.FileCache(filename_to_be_hashed, ignore_cache='nothing', cache_token=None)

Bases: molyso.generic.etc.BaseCache

A caching class which stores the data in flat files.

build_cache_filename(suffix)

Parameters:suffix – Returns:

contains(key)

Parameters:key – Returns:

get(key)

Parameters:key – Returns:

set(key, value)

Parameters:

• key –
• value –

class molyso.generic.etc.NotReallyATree(iterable)

Bases: list

The class is a some-what duck-type compatible (it has a query method) dumb replacement for (c)KDTrees. It can be used to find the nearest matching point to a query point. (And does that by exhaustive search…)

query(q)

Finds the point which is nearest to q. Uses the Euclidean distance.

Parameters:q – query point Returns:distance, index Return type:float, int

>>> t = NotReallyATree([[1.0, 1.0], [2.0, 2.0], [3.0, 3.0]])
>>> t.query([1.25, 1.25])
(0.3535533905932738, 0)
>>> t = NotReallyATree([[1.0, 1.0], [2.0, 2.0], [3.0, 3.0]])
>>> t.query([2.3535533905932737622, 2.3535533905932737622])
(0.5000000000000002, 1)

class molyso.generic.etc.QuickTableDumper(recipient=None)

Bases: object

Parameters:recipient –

add(row)

Parameters:row –

delimiter = '\t'

line_end = '\n'

precision = 8

stringify(obj)

Parameters:obj – Returns:

write_list(the_list)

Parameters:the_list –

class molyso.generic.etc.Sqlite3Cache(*args, **kwargs)

Bases: molyso.generic.etc.BaseCache

A caching class which stores the data in a sqlite3 database.

contains(key)

Parameters:key – Returns:

get(key)

Parameters:key – Returns:

keys()

Returns:

set(key, value)

Parameters:

• key –
• value –

molyso.generic.etc.bits_to_numpy_type(bits)

Returns a numpy.dtype for one of the common image bit-depths: 8 for unsigned int, 16 for unsigned short, 32 for float

Parameters:bits – Returns:

molyso.generic.etc.correct_windows_signal_handlers()

Dummy for non-windows os.

molyso.generic.etc.debug_init()

Initialized debug mode, as of now this means that DebugPlot is set to active (it will produce a debug.pdf)

molyso.generic.etc.dummy_progress_indicator()

Returns:

molyso.generic.etc.fancy_progress_bar(iterable)

Parameters:iterable – Returns:

molyso.generic.etc.ignorant_next(iterable)

Will try to iterate to the next value, or return None if none is available.

Parameters:iterable – Returns:

molyso.generic.etc.iter_time(iterable)

Will print the total time elapsed during iteration of iterable afterwards.

Parameters:iterable (iterable) – iterable Return type:iterable Returns:iterable

molyso.generic.etc.parse_range(s, maximum=0)

Parameters:

• s –
• maximum –

Returns:

molyso.generic.etc.prettify_numpy_array(arr, space_or_prefix)

Returns a properly indented string representation of a numpy array.

Parameters:

• arr –
• space_or_prefix –

Returns:

molyso.generic.etc.silent_progress_bar(iterable)

Dummy function, just returns an iterator.

Parameters:iterable (iterable) – the iterable to turn into an iterable Returns:iterable Return type:iterable

>>> next(silent_progress_bar([1, 2, 3]))
1

molyso.generic.fft module

fft.py contains Fourier transform related helper functions mainly it abstracts the Fourier transform itself (currently just passing the calls through to the numpy functions)

molyso.generic.fft.hires_power_spectrum(signal, oversampling=1)

Return a high resolution power spectrum (compare power_spectrum()) Resolution is enhanced by feeding the FFT a n times larger, zero-padded signal, which will yield frequency values of higher precision.

Parameters:

• signal (numpy.array) – input signal
• oversampling (int) – oversampling factor

Returns:

frequencies and fourier transformed values

Return type:

tuple(numpy.array, numpy.array)

molyso.generic.fft.power_spectrum(signal)

Return a power (absolute/real) spectrum (as opposed to the complex spectrum returned by spectrum() itself)

Parameters:signal (numpy.array) – input signal Returns:frequencies and fourier transformed values Return type:tuple(numpy.array, numpy.array)

molyso.generic.fft.spectrum(signal)

Return a raw spectrum (values are complex). Use power_spectrum() to directly get real values.

Parameters:signal (numpy.array) – input signal Returns:frequencies and fourier transformed values Return type:tuple(numpy.array, numpy.array)

molyso.generic.fft.spectrum_bins(signal)

Returns the bins associated with a Fourier transform of a signal of the same length of signal

Parameters:signal (numpy.array) – input signal Returns:frequency bins Return type:numpy.array

molyso.generic.fft.spectrum_bins_by_length(len_signal)

Returns the bins associated with a Fourier transform of a signal of the length len_signal

Parameters:len_signal (int) – length of the desired bin distribution Returns:frequency bins Return type:numpy.array

molyso.generic.fft.spectrum_fourier(signal)

Calls the Fourier transform, and returns only the first half of the transform results

Parameters:signal (numpy.array) – input signal Returns:Fourier transformed data Return type:numpy.array

molyso.generic.otsu module

otsu.py contains an implementation of Otsu’s thresholding method, taken verbatim from scikit-image.

molyso.generic.otsu.histogram(image, nbins=256)

Return histogram of image.

Unlike numpy.histogram, this function returns the centers of bins and does not rebin integer arrays. For integer arrays, each integer value has its own bin, which improves speed and intensity-resolution.

The histogram is computed on the flattened image: for color images, the function should be used separately on each channel to obtain a histogram for each color channel.

Parameters:

• image (numpy.ndarray) – Input image.
• nbins (int, optional) – Number of bins used to calculate histogram. This value is ignored for integer arrays.

Returns:

The values of the histogram, the values at the center of the bins.

Return type:

tuple(numpy.ndarray, numpy.ndarray)

molyso.generic.otsu.threshold_otsu(image, nbins=256)

Return threshold value based on Otsu’s method.

[1]	Wikipedia, http://en.wikipedia.org/wiki/Otsu’s_Method

Parameters:

• image (numpy.ndarray) – Input image
• nbins (int, optional) – Number of bins used to calculate histogram. This value is ignored for integer arrays.

Returns:

Upper threshold value. All pixels intensities that less or equal of this value assumed as foreground.

Return type:

float

molyso.generic.registration module

registration.py contains a simple 2D image registration function, which registers images by checking the individual horizontal and vertical shifts

molyso.generic.registration.shift_image(image, shift, background='input')

Parameters:

• image –
• shift –
• background –

Returns:

raise ValueError:  

molyso.generic.registration.translation_2x1d(image_a=None, image_b=None, ffts_a=(), ffts_b=(), return_a=False, return_b=False)

Parameters:

• image_a –
• image_b –
• ffts_a –
• ffts_b –
• return_a –
• return_b –

Returns:

molyso.generic.rotation module

molyso.generic.signal module

signal processing helper routines

class molyso.generic.signal.ExtremeAndProminence

Bases: molyso.generic.signal.ExtremeAndProminence

Result of find_extrema_and_prominence call.

Variables:

• maxima –
• minima –
• signal –
• order –
• max_spline –
• min_spline –
• xpts –
• max_spline_points –
• min_spline_points –
• prominence –

molyso.generic.signal.each_image_slice(image, steps, direction='vertical')

Parameters:

• image –
• steps –
• direction –

Returns:

>>> list(each_image_slice(np.ones((4, 4,)), 2, direction='vertical'))
[(0, 2, array([[1., 1.],
       [1., 1.],
       [1., 1.],
       [1., 1.]])), (1, 2, array([[1., 1.],
       [1., 1.],
       [1., 1.],
       [1., 1.]]))]
>>> list(each_image_slice(np.ones((4, 4,)), 2, direction='horizontal'))
[(0, 2, array([[1., 1., 1., 1.],
       [1., 1., 1., 1.]])), (1, 2, array([[1., 1., 1., 1.],
       [1., 1., 1., 1.]]))]

molyso.generic.signal.find_extrema_and_prominence(signal, order=5)

Generates various extra information / signals.

Parameters:

• signal (numpy.ndarray) – input signal
• order (int) – relative minima/maxima order, see other functions

Returns:

an ExtremeAndProminence object with various information members

Return type:

ExtremeAndProminence

>>> result = find_extrema_and_prominence(np.array([1, 2, 3, 2, 1, 0, 1, 2, 15, 2, -15, 2, 1]), 2)
>>> result = result._replace(max_spline=None, min_spline=None)  # just for doctests
>>> result
ExtremeAndProminence(maxima=array([2, 8]), minima=array([ 5, 10]), signal=array([  1,   2,   3,   2,   1,   0,   1,   2,  15,   2, -15,   2,   1]), order=2, max_spline=None, min_spline=None, xpts=array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12.]), max_spline_points=array([ 3.    ,  2.4875,  3.    ,  4.3125,  6.2   ,  8.4375, 10.8   ,
       13.0625, 15.    , 16.3875, 17.    , 16.6125, 15.    ]), min_spline_points=array([-9.73055091e-16,  3.38571429e+00,  4.71428571e+00,  4.35000000e+00,
        2.65714286e+00,  5.21804822e-15, -3.25714286e+00, -6.75000000e+00,
       -1.01142857e+01, -1.29857143e+01, -1.50000000e+01, -1.57928571e+01,
       -1.50000000e+01]), prominence=array([ 3.        , -0.89821429, -1.71428571, -0.0375    ,  3.54285714,
        8.4375    , 14.05714286, 19.8125    , 25.11428571, 29.37321429,
       32.        , 32.40535714, 30.        ]))

molyso.generic.signal.find_insides(signal)

Parameters:signal – Returns:

>>> find_insides(np.array([False, False, True, True, True, False, False, True, True, False, False]))
array([[2, 5],
       [7, 9]])

molyso.generic.signal.find_phase(signal_1=None, signal_2=None, fft_1=None, fft_2=None, return_1=False, return_2=False)

Finds the phase (time shift) between two signals. Either signalX or fftX should be set; the FFTs can be returned in order to cache them locally…

Parameters:

• signal_1 (numpy.ndarray or None) – first input signal
• signal_2 (numpy.ndarray or None) – second input signal
• fft_1 (numpy.ndarray or None) – first input fft
• fft_2 (numpy.ndarray or None) – second input fft
• return_1 (bool) – whether fft1 should be returned
• return_2 (bool) – whether fft2 should be returned

Returns:

(shift, (fft1 if return_1), (fft2 if return_2))

Return type:

tuple

>>> find_phase(np.array([0, 1, 0, 0, 0]), np.array([0, 0, 0, 1, 0]))
(2,)

molyso.generic.signal.fit_to_type(image, new_dtype)

Parameters:

• image –
• new_dtype –

Returns:

>>> fit_to_type(np.array([-7, 4, 18, 432]), np.uint8)
array([  0,   6,  14, 255], dtype=uint8)
>>> fit_to_type(np.array([-7, 4, 18, 432]), np.int8)
array([-128, -121, -113,  127], dtype=int8)
>>> fit_to_type(np.array([-7, 4, 18, 432]), np.bool)
array([False, False, False,  True])
>>> fit_to_type(np.array([-7, 4, 18, 432]), np.float32)
array([ -7.,   4.,  18., 432.], dtype=float32)

molyso.generic.signal.horizontal_mean(image)

Calculates the horizontal mean of an image. Note: Image is assumed HORIZONTAL x VERTICAL.

Parameters:image (numpy.ndarray) – input image Returns:

>>> horizontal_mean(np.array([[ 1,  2,  3,  4],
...                            [ 5,  6,  7,  8],
...                            [ 9, 10, 11, 12],
...                            [13, 14, 15, 16]]))
array([ 7.,  8.,  9., 10.])

molyso.generic.signal.normalize(data)

normalizes the values in arr to 0 - 1

Parameters:data – input array Returns:normalized array

>>> normalize(np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]))
array([0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. ])

>>> normalize(np.array([10, 15, 20]))
array([0. , 0.5, 1. ])

molyso.generic.signal.one_every_n(length, n=1, shift=0)

Parameters:

• length –
• n –
• shift –

Returns:

>>> one_every_n(10, n=2, shift=0)
array([1., 0., 1., 0., 1., 0., 1., 0., 1., 0.])
>>> one_every_n(10, n=2, shift=1)
array([0., 1., 0., 1., 0., 1., 0., 1., 0., 1.])
>>> one_every_n(10, n=1, shift=0)
array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])

molyso.generic.signal.outliers(data, times_std=2.0)

Parameters:

• data –
• times_std –

Returns:

>>> outliers(np.array([10, 9, 11, 40, 8, 12, 14, 7]), times_std=1.0)
array([False, False, False,  True, False, False, False, False])

molyso.generic.signal.relative_maxima(signal, order=1)

Parameters:

• signal –
• order –

Returns:

>>> relative_maxima(np.array([1, 2, 3, 2, 1, 0, 1, 2, 15, 2, -15, 2, 1]), 2)
array([2, 8])

molyso.generic.signal.relative_minima(signal, order=1)

Parameters:

• signal –
• order –

Returns:

>>> relative_minima(np.array([1, 2, 3, 2, 1, 0, 1, 2, 15, 2, -15, 2, 1]), 2)
array([ 5, 10])

molyso.generic.signal.remove_outliers(data, times_std=2.0)

Parameters:

• data –
• times_std –

Returns:

>>> remove_outliers(np.array([10, 9, 11, 40, 8, 12, 14, 7]), times_std=1.0)
array([10,  9, 11,  8, 12, 14,  7])

molyso.generic.signal.simple_baseline_correction(signal, window_width=None)

Performs a simple baseline correction by subtracting a strongly smoothed version of the signal from itself.

Parameters:

• signal – input signal
• window_width – smoothing window width

Returns:

>>> simple_baseline_correction(np.array([10, 11, 12, 11, 10]))
array([-1.        ,  0.375     ,  1.        , -0.375     , -0.96428571])

molyso.generic.signal.threshold_outliers(data, times_std=2.0)

removes outliers

Parameters:

• data –
• times_std –

Returns:

>>> threshold_outliers(np.array([10, 9, 11, 40, 8, 12, 14, 7]), times_std=1.0)
array([10,  9, 11, 20,  8, 12, 14,  7])

molyso.generic.signal.vertical_mean(image)

Calculates the vertical mean of an image. Note: Image is assumed HORIZONTAL x VERTICAL.

Parameters:image – Returns:

>>> vertical_mean(np.array([[ 1,  2,  3,  4],
...                            [ 5,  6,  7,  8],
...                            [ 9, 10, 11, 12],
...                            [13, 14, 15, 16]]))
array([ 2.5,  6.5, 10.5, 14.5])

molyso.generic.smoothing module

smoothing.py contains the main smoothing function, which works by convolving a signal with a smoothing kernel, a signals function which acts as a cache for kernels, as well as the hamming_smooth function, which is the only one currently used by external files, providing a simplified interface for smoothing with hamming kernels.

molyso.generic.smoothing.hamming_smooth(signal, window_width, no_cache=False)

Smooths a signal by convolving with a hamming window of given width. Caches by the hamming windows by default.

Parameters:

• signal (numpy.ndarray) – input signal to be smoothed
• window_width (int) – window width for the hamming kernel
• no_cache (bool) – default False, disables caching, e.g., for non-standard window sizes

Returns:

the smoothed signal

Return type:

numpy.ndarray

>>> hamming_smooth(np.array([0, 0, 0, 0, 1, 0, 0, 0, 0]), 3)
array([0.        , 0.        , 0.        , 0.        , 0.06896552,
       0.86206897, 0.06896552, 0.        , 0.        ])

molyso.generic.smoothing.signals(function, parameters)

Signal cache helper function. Either retrieves or creates and stores a signal which can be created by calling the given function with the given parameters.

Parameters:

• function (callable) – Window function to be called
• parameters (*any) – Parameters to be passed to the function

Returns:

function(*parameters)

Return type:

dependent on function

>>> signals(np.ones, 3)
array([1., 1., 1.])

molyso.generic.smoothing.smooth(signal, kernel)

Generic smoothing function, smooths by convolving one signal with another.

Parameters:

• signal (numpy.ndarray) – input signal to be smoothed
• kernel (numpy.ndarray) – smoothing kernel to be used. will be normalized to \(\sum=1\)

Returns:

The signal convolved with the kernel

Return type:

numpy.ndarray

>>> smooth(np.array([0, 0, 0, 0, 1, 0, 0, 0, 0]), np.ones(3))
array([0.        , 0.        , 0.        , 0.        , 0.33333333,
       0.33333333, 0.33333333, 0.        , 0.        ])

molyso.generic.tunable module

tunable.py contains a tunable (settings to be changed depending on input data) management class

class molyso.generic.tunable.TunableManager

Bases: object

Static object handling tunables (that is, parameters which are user-changeable)

Tunables have default values, which must be set as a parameter by the function asking for the tunable. That way, default configuration is inlined, and does not need to be centrally managed. In order to collect all defaults, a typical run of the program has to be performed, and the collected default values to be dumped afterwards.

Variables:

• defaults (dict) – Internal map of defaults
• current (dict) – Current tunables, with possible overrides.
• force_default (bool) – Whether to force usage of default values (default: False)

current = {}

defaults = {}

descriptions = {}

force_default = False

classmethod get_defaults()

Gets the defaults, which were collected during the calls asking for various tunables.

Returns:either the overridden tunable or the default value Return type:dependent on default

>>> TunableManager.defaults = {}
>>> value = tunable('my.tunable', 3.1415)
>>> TunableManager.get_defaults()
{'my.tunable': 3.1415}

classmethod get_descriptions()

Gets descriptions.

Returns:The descriptions. Return type:dict

classmethod get_table()

classmethod get_tunable(what, default)

Returns either an overridden tunable, or the default value. The result will be casted to the type of default.

Parameters:

• what (str) – tunable to look up
• default – default value

Returns:

either the overridden tunable or the default value

>>> tunable('my.tunable', 3.1415)
3.1415

classmethod load_tunables(data)

Sets the tunables.

Parameters:data (dict) – set of tunables to load Return type:None

>>> TunableManager.load_tunables({'foo': 'bar'})
>>> tunable('foo', 'not bar')
'bar'

logger = <Logger molyso.generic.tunable.TunableManager (WARNING)>

classmethod set_description(what, description)

Sets a description for a paremeter.

Parameters:

• what – parameter to describe
• description – description

Returns:

molyso.generic.tunable.tunable(what, default, description=None)

Syntactic sugar helper function, to quickly get a tunable. Calls: TunableManager.get_tunable(what, default)

Parameters:

• what (str or unicode) – tunable to look up
• default – default value
• description – description

Returns:

either the overridden tunable or the default value

Return type:

type(default)

>>> tunable('my.tunable', 3.1415)
3.1415