molyso.generic package¶
Generic module. Collection of various helper functionality, from library routines for signal processing, to os specific workarounds.
molyso.generic.etc module¶
etc.py contains various helper functions and classes, which are not directly related to data processing.
-
class
molyso.generic.etc.
BaseCache
(filename_to_be_hashed, ignore_cache='nothing', cache_token=None)[source]¶ Bases:
object
A caching class
-
molyso.generic.etc.
Cache
¶ alias of
molyso.generic.etc.FileCache
-
class
molyso.generic.etc.
FileCache
(filename_to_be_hashed, ignore_cache='nothing', cache_token=None)[source]¶ Bases:
molyso.generic.etc.BaseCache
A caching class which stores the data in flat files.
-
class
molyso.generic.etc.
NotReallyATree
(iterable)[source]¶ Bases:
list
The class is a some-what duck-type compatible (it has a
query
method) dumb replacement for (c)KDTrees. It can be used to find the nearest matching point to a query point. (And does that by exhaustive search…)-
query
(q)[source]¶ Finds the point which is nearest to
q
. Uses the Euclidean distance.Parameters: q – query point Returns: distance, index Return type: float, int >>> t = NotReallyATree([[1.0, 1.0], [2.0, 2.0], [3.0, 3.0]]) >>> t.query([1.25, 1.25]) (0.3535533905932738, 0) >>> t = NotReallyATree([[1.0, 1.0], [2.0, 2.0], [3.0, 3.0]]) >>> t.query([2.3535533905932737622, 2.3535533905932737622]) (0.5000000000000002, 1)
-
-
class
molyso.generic.etc.
QuickTableDumper
(recipient=None)[source]¶ Bases:
object
Parameters: recipient – -
delimiter
= '\t'¶
-
line_end
= '\n'¶
-
precision
= 8¶
-
-
class
molyso.generic.etc.
Sqlite3Cache
(*args, **kwargs)[source]¶ Bases:
molyso.generic.etc.BaseCache
A caching class which stores the data in a sqlite3 database.
-
molyso.generic.etc.
bits_to_numpy_type
(bits)[source]¶ Returns a numpy.dtype for one of the common image bit-depths: 8 for unsigned int, 16 for unsigned short, 32 for float
Parameters: bits – Returns:
-
molyso.generic.etc.
debug_init
()[source]¶ Initialized debug mode, as of now this means that DebugPlot is set to active (it will produce a debug.pdf)
-
molyso.generic.etc.
ignorant_next
(iterable)[source]¶ Will try to iterate to the next value, or return None if none is available.
Parameters: iterable – Returns:
-
molyso.generic.etc.
iter_time
(iterable)[source]¶ Will print the total time elapsed during iteration of
iterable
afterwards.Parameters: iterable (iterable) – iterable Return type: iterable Returns: iterable
molyso.generic.fft module¶
fft.py contains Fourier transform related helper functions mainly it abstracts the Fourier transform itself (currently just passing the calls through to the numpy functions)
-
molyso.generic.fft.
hires_power_spectrum
(signal, oversampling=1)[source]¶ Return a high resolution power spectrum (compare
power_spectrum()
) Resolution is enhanced by feeding the FFT a n times larger, zero-padded signal, which will yield frequency values of higher precision.Parameters: - signal (numpy.array) – input signal
- oversampling (int) – oversampling factor
Returns: frequencies and fourier transformed values
Return type: tuple(numpy.array, numpy.array)
-
molyso.generic.fft.
power_spectrum
(signal)[source]¶ Return a power (absolute/real) spectrum (as opposed to the complex spectrum returned by
spectrum()
itself)Parameters: signal (numpy.array) – input signal Returns: frequencies and fourier transformed values Return type: tuple(numpy.array, numpy.array)
-
molyso.generic.fft.
spectrum
(signal)[source]¶ Return a raw spectrum (values are complex). Use
power_spectrum()
to directly get real values.Parameters: signal (numpy.array) – input signal Returns: frequencies and fourier transformed values Return type: tuple(numpy.array, numpy.array)
-
molyso.generic.fft.
spectrum_bins
(signal)[source]¶ Returns the bins associated with a Fourier transform of a signal of the same length of signal
Parameters: signal (numpy.array) – input signal Returns: frequency bins Return type: numpy.array
molyso.generic.otsu module¶
otsu.py contains an implementation of Otsu’s thresholding method, taken verbatim from scikit-image.
-
molyso.generic.otsu.
histogram
(image, nbins=256)[source]¶ Return histogram of image.
Unlike numpy.histogram, this function returns the centers of bins and does not rebin integer arrays. For integer arrays, each integer value has its own bin, which improves speed and intensity-resolution.
The histogram is computed on the flattened image: for color images, the function should be used separately on each channel to obtain a histogram for each color channel.
Parameters: - image (numpy.ndarray) – Input image.
- nbins (int, optional) – Number of bins used to calculate histogram. This value is ignored for integer arrays.
Returns: The values of the histogram, the values at the center of the bins.
Return type: tuple(numpy.ndarray, numpy.ndarray)
-
molyso.generic.otsu.
threshold_otsu
(image, nbins=256)[source]¶ Return threshold value based on Otsu’s method.
[1] Wikipedia, http://en.wikipedia.org/wiki/Otsu’s_Method Parameters: - image (numpy.ndarray) – Input image
- nbins (int, optional) – Number of bins used to calculate histogram. This value is ignored for integer arrays.
Returns: Upper threshold value. All pixels intensities that less or equal of this value assumed as foreground.
Return type: float
molyso.generic.registration module¶
registration.py contains a simple 2D image registration function, which registers images by checking the individual horizontal and vertical shifts
molyso.generic.rotation module¶
molyso.generic.signal module¶
signal processing helper routines
-
class
molyso.generic.signal.
ExtremeAndProminence
[source]¶ Bases:
molyso.generic.signal.ExtremeAndProminence
Result of find_extrema_and_prominence call.
Variables: - maxima –
- minima –
- signal –
- order –
- max_spline –
- min_spline –
- xpts –
- max_spline_points –
- min_spline_points –
- prominence –
-
molyso.generic.signal.
each_image_slice
(image, steps, direction='vertical')[source]¶ Parameters: - image –
- steps –
- direction –
Returns: >>> list(each_image_slice(np.ones((4, 4,)), 2, direction='vertical')) [(0, 2, array([[1., 1.], [1., 1.], [1., 1.], [1., 1.]])), (1, 2, array([[1., 1.], [1., 1.], [1., 1.], [1., 1.]]))] >>> list(each_image_slice(np.ones((4, 4,)), 2, direction='horizontal')) [(0, 2, array([[1., 1., 1., 1.], [1., 1., 1., 1.]])), (1, 2, array([[1., 1., 1., 1.], [1., 1., 1., 1.]]))]
-
molyso.generic.signal.
find_extrema_and_prominence
(signal, order=5)[source]¶ Generates various extra information / signals.
Parameters: - signal (numpy.ndarray) – input signal
- order (int) – relative minima/maxima order, see other functions
Returns: an ExtremeAndProminence object with various information members
Return type: >>> result = find_extrema_and_prominence(np.array([1, 2, 3, 2, 1, 0, 1, 2, 15, 2, -15, 2, 1]), 2) >>> result = result._replace(max_spline=None, min_spline=None) # just for doctests >>> result ExtremeAndProminence(maxima=array([2, 8]), minima=array([ 5, 10]), signal=array([ 1, 2, 3, 2, 1, 0, 1, 2, 15, 2, -15, 2, 1]), order=2, max_spline=None, min_spline=None, xpts=array([ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12.]), max_spline_points=array([ 3. , 2.4875, 3. , 4.3125, 6.2 , 8.4375, 10.8 , 13.0625, 15. , 16.3875, 17. , 16.6125, 15. ]), min_spline_points=array([-9.73055091e-16, 3.38571429e+00, 4.71428571e+00, 4.35000000e+00, 2.65714286e+00, 5.21804822e-15, -3.25714286e+00, -6.75000000e+00, -1.01142857e+01, -1.29857143e+01, -1.50000000e+01, -1.57928571e+01, -1.50000000e+01]), prominence=array([ 3. , -0.89821429, -1.71428571, -0.0375 , 3.54285714, 8.4375 , 14.05714286, 19.8125 , 25.11428571, 29.37321429, 32. , 32.40535714, 30. ]))
-
molyso.generic.signal.
find_insides
(signal)[source]¶ Parameters: signal – Returns: >>> find_insides(np.array([False, False, True, True, True, False, False, True, True, False, False])) array([[2, 5], [7, 9]])
-
molyso.generic.signal.
find_phase
(signal_1=None, signal_2=None, fft_1=None, fft_2=None, return_1=False, return_2=False)[source]¶ Finds the phase (time shift) between two signals. Either signalX or fftX should be set; the FFTs can be returned in order to cache them locally…
Parameters: - signal_1 (numpy.ndarray or None) – first input signal
- signal_2 (numpy.ndarray or None) – second input signal
- fft_1 (numpy.ndarray or None) – first input fft
- fft_2 (numpy.ndarray or None) – second input fft
- return_1 (bool) – whether fft1 should be returned
- return_2 (bool) – whether fft2 should be returned
Returns: (shift, (fft1 if return_1), (fft2 if return_2))
Return type: tuple
>>> find_phase(np.array([0, 1, 0, 0, 0]), np.array([0, 0, 0, 1, 0])) (2,)
-
molyso.generic.signal.
fit_to_type
(image, new_dtype)[source]¶ Parameters: - image –
- new_dtype –
Returns: >>> fit_to_type(np.array([-7, 4, 18, 432]), np.uint8) array([ 0, 6, 14, 255], dtype=uint8) >>> fit_to_type(np.array([-7, 4, 18, 432]), np.int8) array([-128, -121, -113, 127], dtype=int8) >>> fit_to_type(np.array([-7, 4, 18, 432]), np.bool) array([False, False, False, True]) >>> fit_to_type(np.array([-7, 4, 18, 432]), np.float32) array([ -7., 4., 18., 432.], dtype=float32)
-
molyso.generic.signal.
horizontal_mean
(image)[source]¶ Calculates the horizontal mean of an image. Note: Image is assumed HORIZONTAL x VERTICAL.
Parameters: image (numpy.ndarray) – input image Returns: >>> horizontal_mean(np.array([[ 1, 2, 3, 4], ... [ 5, 6, 7, 8], ... [ 9, 10, 11, 12], ... [13, 14, 15, 16]])) array([ 7., 8., 9., 10.])
-
molyso.generic.signal.
normalize
(data)[source]¶ normalizes the values in arr to 0 - 1
Parameters: data – input array Returns: normalized array >>> normalize(np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])) array([0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. ])
>>> normalize(np.array([10, 15, 20])) array([0. , 0.5, 1. ])
-
molyso.generic.signal.
one_every_n
(length, n=1, shift=0)[source]¶ Parameters: - length –
- n –
- shift –
Returns: >>> one_every_n(10, n=2, shift=0) array([1., 0., 1., 0., 1., 0., 1., 0., 1., 0.]) >>> one_every_n(10, n=2, shift=1) array([0., 1., 0., 1., 0., 1., 0., 1., 0., 1.]) >>> one_every_n(10, n=1, shift=0) array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])
-
molyso.generic.signal.
outliers
(data, times_std=2.0)[source]¶ Parameters: - data –
- times_std –
Returns: >>> outliers(np.array([10, 9, 11, 40, 8, 12, 14, 7]), times_std=1.0) array([False, False, False, True, False, False, False, False])
-
molyso.generic.signal.
relative_maxima
(signal, order=1)[source]¶ Parameters: - signal –
- order –
Returns: >>> relative_maxima(np.array([1, 2, 3, 2, 1, 0, 1, 2, 15, 2, -15, 2, 1]), 2) array([2, 8])
-
molyso.generic.signal.
relative_minima
(signal, order=1)[source]¶ Parameters: - signal –
- order –
Returns: >>> relative_minima(np.array([1, 2, 3, 2, 1, 0, 1, 2, 15, 2, -15, 2, 1]), 2) array([ 5, 10])
-
molyso.generic.signal.
remove_outliers
(data, times_std=2.0)[source]¶ Parameters: - data –
- times_std –
Returns: >>> remove_outliers(np.array([10, 9, 11, 40, 8, 12, 14, 7]), times_std=1.0) array([10, 9, 11, 8, 12, 14, 7])
-
molyso.generic.signal.
simple_baseline_correction
(signal, window_width=None)[source]¶ Performs a simple baseline correction by subtracting a strongly smoothed version of the signal from itself.
Parameters: - signal – input signal
- window_width – smoothing window width
Returns: >>> simple_baseline_correction(np.array([10, 11, 12, 11, 10])) array([-1. , 0.375 , 1. , -0.375 , -0.96428571])
-
molyso.generic.signal.
threshold_outliers
(data, times_std=2.0)[source]¶ removes outliers
Parameters: - data –
- times_std –
Returns: >>> threshold_outliers(np.array([10, 9, 11, 40, 8, 12, 14, 7]), times_std=1.0) array([10, 9, 11, 20, 8, 12, 14, 7])
-
molyso.generic.signal.
vertical_mean
(image)[source]¶ Calculates the vertical mean of an image. Note: Image is assumed HORIZONTAL x VERTICAL.
Parameters: image – Returns: >>> vertical_mean(np.array([[ 1, 2, 3, 4], ... [ 5, 6, 7, 8], ... [ 9, 10, 11, 12], ... [13, 14, 15, 16]])) array([ 2.5, 6.5, 10.5, 14.5])
molyso.generic.smoothing module¶
smoothing.py contains the main smoothing function, which works by convolving a signal with a smoothing kernel, a signals function which acts as a cache for kernels, as well as the hamming_smooth function, which is the only one currently used by external files, providing a simplified interface for smoothing with hamming kernels.
-
molyso.generic.smoothing.
hamming_smooth
(signal, window_width, no_cache=False)[source]¶ Smooths a signal by convolving with a hamming window of given width. Caches by the hamming windows by default.
Parameters: - signal (numpy.ndarray) – input signal to be smoothed
- window_width (int) – window width for the hamming kernel
- no_cache (bool) – default False, disables caching, e.g., for non-standard window sizes
Returns: the smoothed signal
Return type: numpy.ndarray
>>> hamming_smooth(np.array([0, 0, 0, 0, 1, 0, 0, 0, 0]), 3) array([0. , 0. , 0. , 0. , 0.06896552, 0.86206897, 0.06896552, 0. , 0. ])
-
molyso.generic.smoothing.
signals
(function, parameters)[source]¶ Signal cache helper function. Either retrieves or creates and stores a signal which can be created by calling the given function with the given parameters.
Parameters: - function (callable) – Window function to be called
- parameters (*any) – Parameters to be passed to the function
Returns: function(*parameters)
Return type: dependent on function
>>> signals(np.ones, 3) array([1., 1., 1.])
-
molyso.generic.smoothing.
smooth
(signal, kernel)[source]¶ Generic smoothing function, smooths by convolving one signal with another.
Parameters: - signal (numpy.ndarray) – input signal to be smoothed
- kernel (numpy.ndarray) – smoothing kernel to be used. will be normalized to \(\sum=1\)
Returns: The signal convolved with the kernel
Return type: numpy.ndarray
>>> smooth(np.array([0, 0, 0, 0, 1, 0, 0, 0, 0]), np.ones(3)) array([0. , 0. , 0. , 0. , 0.33333333, 0.33333333, 0.33333333, 0. , 0. ])
molyso.generic.tunable module¶
tunable.py contains a tunable (settings to be changed depending on input data) management class
-
class
molyso.generic.tunable.
TunableManager
[source]¶ Bases:
object
Static object handling tunables (that is, parameters which are user-changeable)
Tunables have default values, which must be set as a parameter by the function asking for the tunable. That way, default configuration is inlined, and does not need to be centrally managed. In order to collect all defaults, a typical run of the program has to be performed, and the collected default values to be dumped afterwards.
Variables: - defaults (dict) – Internal map of defaults
- current (dict) – Current tunables, with possible overrides.
- force_default (bool) – Whether to force usage of default values (default: False)
-
current
= {}¶
-
defaults
= {}¶
-
descriptions
= {}¶
-
force_default
= False¶
-
classmethod
get_defaults
()[source]¶ Gets the defaults, which were collected during the calls asking for various tunables.
Returns: either the overridden tunable or the default value Return type: dependent on default >>> TunableManager.defaults = {} >>> value = tunable('my.tunable', 3.1415) >>> TunableManager.get_defaults() {'my.tunable': 3.1415}
-
classmethod
get_descriptions
()[source]¶ Gets descriptions.
Returns: The descriptions. Return type: dict
-
classmethod
get_tunable
(what, default)[source]¶ Returns either an overridden tunable, or the default value. The result will be casted to the type of default.
Parameters: - what (str) – tunable to look up
- default – default value
Returns: either the overridden tunable or the default value
>>> tunable('my.tunable', 3.1415) 3.1415
-
classmethod
load_tunables
(data)[source]¶ Sets the tunables.
Parameters: data (dict) – set of tunables to load Return type: None >>> TunableManager.load_tunables({'foo': 'bar'}) >>> tunable('foo', 'not bar') 'bar'
-
logger
= <Logger molyso.generic.tunable.TunableManager (WARNING)>¶
-
molyso.generic.tunable.
tunable
(what, default, description=None)[source]¶ Syntactic sugar helper function, to quickly get a tunable. Calls:
TunableManager.get_tunable(what, default)
Parameters: - what (str or unicode) – tunable to look up
- default – default value
- description – description
Returns: either the overridden tunable or the default value
Return type: type(default)
>>> tunable('my.tunable', 3.1415) 3.1415