PD Stefan Bosse - AFEML - Module A: Data and Data Features -

Data and Data Features

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data

Data

In general, data and their values can be divided into:

• Scalar values, such as temperature, age, etc.
• Series of scalar values, such as time series
• Vector and matrix values such as images
• Composite data, i.e. data structures (records)
• Temporal-spatial data, i.e. time-dependent spatial data series, D={D(p,t)={d(p)_i}} with i = {1,2,3,..,t}, p=⟨x,y,..⟩

Data have dimensionality 𝕏^N

• The values of 𝕏 are a dimension from the discrete number set ℕ, real number set ℝ, and the time scale 𝕋 or any categorical value sets 𝕊 (or subsets thereof), e.g., 𝕏=ℝ × ℝ × ℕ.

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data Reduction

Data Reduction

• The aim of data analysis is to reduce input data in terms of size and dimensionality:

$$P(X^N): X^N \rightarrow Y^M\\ |Y|<|X|, M<N$$

Materials science, metrology, and construction engineering uses:

• Commonly metric input variables;
• Often metric or categorical output variables (incl. Boolean variables)

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data Reduction

Data Reduction

function isRaining(temp,sunrad,moisture) = {
  if (temp < 0)                    FALSE
  else if (temp > 40)              FALSE
  else if ((sunrad-moisture) > 30) FALSE
  else                             TRUE
}

A R example from measurement technology with a data reduction function ℝ³ → 𝔹

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data classes

Data classes

Numerical and Metric values

These are values that are countable and where you can meaningfully define relations (such as smaller or larger), i.e. for all real and integers.

• Examples: temperature, length, density, pore size, elongation, force, location, time

Categorical values

These are symbolic values for which either no (meaningful) order relation exists or where at least no differences can be formed.

• Examples: nationality, color names (red < yellow???), Damage type, characteristic feature (anomaly?)

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data classes

m = 1
m = [1.0,1.5,2.5]
c = 'A'
c = ['A','B','A']
c = [TRUE,FALSE,TRUE]
c = factor(m,levels=[1,1.5,2,2.5],labels=['A','B','C','D'])

R examples of numerical and categorical values and conversion (factorization)

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data classes

Scaling of numerical values

Interval scaled

For this type of attributes, only differences (addition or subtraction) make sense. For example, the temperature measured in °C or °F is interval scaled. If it is 20 °C on one day and 10 °C on the following day, it makes sense to talk about a temperature drop of 10 °C, but it does not make sense to say that it is twice as cold as the day before (C(K)∼K, but F(K)/∼K!!).

Ratio scaled

Here you can calculate both differences and ratios between values. For example, for age, one can say that someone who is 20 years old is twice as old as someone who is 10 years old, and 20 is > 10.

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data classes

Order relations

Nominal

The attribute values in the domain are unordered and therefore only equality comparisons make sense. That is, we can only check whether the value of the attribute is the same for two specific instances or not. For example, gender is a nominal attribute.

Ordinal

The attribute values are ordered and thus equality comparisons (is one value equal to another?) and relational comparisons (is one value smaller or larger than another?) are allowed, although it may not be possible to quantify the difference between the values!

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data Aggregations

Data Aggregations

v = c(4)            v = [1.0,1.5,2.5]
v[1] = 1.2
l = list(a=1,b=2)   l = {a=1,b=2}   l={1.0,1.5,2.5}
l$a = 9
m = matrix(0,nrow=2,ncol=3)
m = [1,2,3;4,5,6]
a = array(0,dim=[3,2,4])
df = data.frame(a={1,2,3},b={3,4,5})

R examples of aggregated data)

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data classes (longitudinal)

Data classes (longitudinal)

• Sensor and measurement data variables (both categorical and metric) can be further distinguished in:

Static

The variable s is not variable in time or is to be regarded as stationary (immutable) in a significant time interval t ∈ [t₀, t₁].

Dynamic

The variable s(t) is time-dependent and forms a data series (or time vector) s(t)={s₀,s₁,..s_t} in the case of discrete acquisition, i.e., we are talking about longitudinal data.

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data

Data

Data sets as matrices

• Data can be represented in matrix form as matrix D (analogy to table form) [1]:

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Data

• The vector X is the set of all variables X_i and represent the columns of the matrix D:

$$\vec{X} = (X_{1},X_{2},..,X_{d})$$

• Each row x_j is a record of the variable set X={X_i|i=1,d} with values x and represent an individual example, instance, experiment, entitie, object, and feature vector as a d-digit tuple, depending on the application and objective:

$$\vec{d}_j = \vec{x}_j = (x_{j,1},x_{j,2},..,x_{j,d})$$

df = data.frame(
  X1={'x1,1','x1,2','...'},
  X2={'x2,1','x2,2','...'},
  X3={'x3,1','x3,2','...'}
)
print(df)
      X1      X2     X3 == X
1  "x1,1" "x2,1" "x3,1"
2  "x1,2" "x2,2" "x3,2"
3  "..."  "..."  "..."

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Input and Output Variables

Input and Output Variables

• The variable set is composed of input and output variables: X_xy=X ∩ Y
• Sensors are commonly input variables X
• Statements are output variables Y, i.e. results that can be derived from the input variables (by a function F):

$$\vec{X}_{xy} = (X_{1},X_{2},..,X_{u},Y_{1},Y_{2},..,Y_{v}) \\ \vec{X} = (X_{1},X_{2},..,X_{u}) \\ \vec{Y} = (Y_{1},Y_{2},..,Y_{v}) \\ \vec{d}_j = (x_{j,1},x_{j,2},..,x_{j,u},y_{j,1},y_{j,2},..,y_{j,v}) \\ F(\vec{X}): \vec{X} \rightarrow \vec{Y},$$

with u+v=d.

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Example of a data matrix

Example of a data matrix

• Botanical data set with geometric (numerical) properties of a plant and categorical classification:

1

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Example of a data matrix

• Measurement data set

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Example of a data matrix

tt = data.frame(
  Strain = [0.0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8],
  Force  = [0.0,0.2,0.7,1.5,1.7,1.9,2.0,0.2,-0.5]
)

Measure data stored in a R data.frame

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Example of a data matrix

Attributes

• The measured variables X₁ to X₄ are metric data variables, the variable X₅=y is a categorical variable!

• The measured variables X₁ to X₄ (i.e. sensors) are called attributes because they are properties and descriptive variables of the target variable y.

High-dimensional Data

• Images I=I(x,y[,z]) are commonly two- or three-dimensional spatial data, organised in rows and columns (and levels)
• Spatiotemporal data T=T(x,y[,z],t) is commonly three- or four dimensional and organised in rows, columns (levels), and discrete time points t.

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Sensors

Sensors

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Sensor

Sensor

• Measurement

  • Physical quantities such as temperature, strain, stress, time, absorption
  • Merged survey variables (e.g. ensemble mean values, outliers, ..)

• When measuring with sensors, a distinction is made between:

  • Single or single measurements (single shot)
  • Repeated measurements of the same physical quantity (averaging..)
  • Series of measured values, especially time-resolved data series:
    D = {d₁,d₂,..,d_n}, where commonly Δt(d_i,d_i+1) is constant

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Sensor

Sensor

• Socio-technical systems, surveys

  • Survey variables (answers to questions) are sensors of individual people
  • Merged survey variables (e.g. ensemble mean values) are sensors of groups of people

• Generally available data

  • Social networks and social media
  • Databases of authorities, etc.

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Sensor model

Sensor model

• A sensor is a transducer (indicator for a property that is not directly measurable)

• A sensor therefore generally maps a physical quantity x to another quantity y:

$$S(x): x \rightarrow y,K:correct(x\rightarrow y)$$

• There is usually a calibration function ${K}{\left({f},{x},{y}\right)}$

• Examples are:

  • Pressure → Voltage, Radiation → current, etc.
  • Social networking → Numerical radius value, votes → Politics, i.e., Assignment of numbers to objects or events according to established rules

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Sensor data

Sensor data

• Sensors S are data sources d of physical, sociological or other natural variables x that cannot be detected directly

• The data values (numeric) will be in a definable interval

• Knowledge of the value interval is important for later data processing, analysis, and machine learning!
• Categorical values are also defined by a set

$$S(x): x \rightarrow d \\ d \in [a,b] \Rightarrow \{v_0,v_1,..,v_i\}$$

PD Stefan Bosse - AFEML - Module A: Data and Data Features - Sensor data

PD Stefan Bosse - AFEML - Module A: Measurement and sensory systems - Sensor data

Measurement and sensory systems

PD Stefan Bosse - AFEML - Module A: Measurement and sensory systems - Measurement methods

Measurement methods

A distinction is made between two different measurement methods:

Passive measuring method (P)

The sensory values are the result of an intrinsic property (e.g., density) or already existing external variables (temperature). The stimulus of the measurement is the component, the person, the environment.

Active measurement methods (A)

There is an active stimulus whose response signal is detected by the sensor. An example is the ultrasonic measurement method with guided waves. The sensor signal is always dependent on the stimulus. In sociology, for example, the stimulus is a catalog of questions in a survey, the answers are the sensor variables.

PD Stefan Bosse - AFEML - Module A: Measurement and sensory systems - Measurement methods

PD Stefan Bosse - AFEML - Module A: Measurement and sensory systems - Measurement methods

PD Stefan Bosse - AFEML - Module A: Measurement and sensory systems - Measurement methods

PD Stefan Bosse - AFEML - Module A: Signal Features - Measurement methods

Signal Features

1. Statistical Features

2. Spatial Features (Images, geometric features)

3. Frequency and spectral Features /time and space)

4. Differences to reference signals

5. Transformed Signals

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

Assumption: Data series

• But any image can be transformed into a pixel data series, too!
• Any column of a data table is a data series (but independent values and unordered!)

There is a data series d related to one variable x(from sensor s):

$$\vec{{d}}={\left\lbrace{d}_{{1}},{d}_{{2}},\ldots,{d}_{{n}}\right\rbrace},{s}:{x}\to{d}$$

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

... and many more

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

use math
Force  = [0.0,0.2,0.7,1.5,1.7,1.9,2.0,0.2,-0.5]
statsForce = fivenum(Force)
statsForce$std = sd(Force)
cprint(statsForce)
{min : -0.5 , q1 : 0.2 , median : 0.7 , mean : 0.855 , 
 q3 : 1.7 , max : 2, sd: 0.93}

Statistical analysis of data series or vectors in R

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

... and many more

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

use math
Force  = [0.0,0.2,0.7,1.5,1.7,1.9,2.0,0.2,-0.5]
mn = moment(Force,order=2,central=TRUE)
print(mn)

Higher order moment analysis of data series or vectors in R

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

Meaning of higher order moments (Wikipedia)

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

PD Stefan Bosse - AFEML - Module A: Signal Features - Statistical Features

Statistical Features

• Finally, all statistical features create a new input vector (for ML) X^f derived from the original input variables X:

$$\text{Stat}{\left({X}\right)}:{X}\to{X}^{{f}}\\ {X}={\left({X}_{{1}},..,{X}_{{i}}\right)},{X}^{{f}}={\left({{X}_{{1}}^{{f}}},..,{{X}_{{j}}^{{f}}}\right)}\text{},{i}\gg{j}$$

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Image Features

Low Level

1. Histogram H(I)={h₁,..,h_k}, where each histogram variable represents the number of pixels within an intensity interval [i,i+Δ] (can be split into separate RGB histograms for colour images)
2. Average (mean) intensity I, noise (intensity distribution statistics)
3. Extrema intensities min(I), max(I)
4. Frequency spectrum F(I)={f₁,..,f_s}, where each frequency represents a wavenumber in the wave room
5. Intensity gradients and profiles along lines (axis)
6. Addition and subtraction of images (using, e.g., base-line reference images)

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Image Features

High Level

1. Intensity gradients
2. Edges
3. Geometrical figures
4. Object clusters
5. Regions-of-interest (ROI), defined by bounding boxes or closed polygons
6. Labelled and classified ROIs
7. Feature point markings
8. Threshold Binarization (dimensionality reduction and feature amplification)

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Transformations

A simple way to reduce the dimension of our feature vector is to decrease the size of the image with decimation (downsampling) by reducing the resolution of the image.

• If the color component is not relevant, we can also convert pictures to grayscale to divide the number dimension by three.

• Intensity homogenisation using transfer functions

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Color Spaces

1. RGB: Three channels per pixel for each color R(ed), G(reen), B(lue) providing the color intensity
2. RGBA: RGB with an additional alpha (tranparency) channel
3. Grayscale: One channel per pixel providing the intensity (average or luminescence)

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Color Spaces

1. Average RGB ⇒ Grayscale transformation

$${I}{\left({x},{y}\right)}=\frac{{{R}{\left({x},{y}\right)}+{G}{\left({x},{y}\right)}+{B}{\left({x},{y}\right)}}}{{3}}$$

2. More natural color weighted luma RGB ⇒ Grayscale transformation

$${I}{\left({x},{y}\right)}={0.299}{R}{\left({x},{y}\right)}+{0.587}{G}{\left({x},{y}\right)}+{0.114}{B}{\left({x},{y}\right)}$$

3. RGBA ⇒ Grayscale transformation

$${I}{\left({x},{y}\right)}=\frac{{{f{{\left({R}{\left({x},{y}\right)}\right)}}}+{f{{\left({G}{\left({x},{y}\right)}\right)}}}+{f{{\left({B}{\left({x},{y}\right)}\right)}}}}}{{3}}\\ {f{{\left({i},{a}\right)}}}={\left({1}-{a}\right)}_{{k}}+{a}_{{i}}\\ {a}=\frac{{{A}{\left({x},{y}\right)}}}{{k}}\\ {k}={255}$$

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Look-up Tables

• A look-up table can be considered as a discrete mapping function f(x): x → y, whereby the index, i.e,, a specific row, is given by the (discrete) x value, and y is the value in the specific row.

• Only meaningful for small and discrete intensity value ranges, e.g., 8 Bit [0,255]

• Only rough approximation of an intensity transfer function with continous value distributions, but fast method!

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Look-up Tables

use plot,math,imager
vals = [1,3,5,6,7.5,8,8.5,9,9.5,10]
mylut = lut(vals,range=[0,9])
img = matrix(runif(100)*10,10,10)
img.isca = mylut(img)
plot(img,auto.scale=TRUE)
hist(img,breaks=20)
plot(img.isca,auto.scale=TRUE)
hist(img.isca,breaks=20)

LUT function in R(+) applied to a random matrix

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Histogram of Oriented Gradient

The HOG feature descriptor is a popular technique used in computer vision and image processing for detecting objects in digital images.

The HOG descriptor is a type of feature descriptor that encodes the shape and appearance of an object by computing the distribution of intensity gradients in an image.

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Histogram of an Image

use math,plot
img = matrix(runif(100),10,10)
plot(img,auto.scale=TRUE)
hist(img,ylim=[0,1])
img[img>0.5]=1
plot(img,auto.scale=TRUE)
hist(img,ylim=[0,1])

Histogram of a uniformly distributed random image and image binarization

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Intensity Homogenization

• Image processing and transformation algorithms can be sensitive to intensity inhomogeneity.

• Algorithms:

  • Histogram Equalization (HE), Brightness Preserving Bi-Histogram Equalization (BBHE)
  • Geometrical Image Intensity Equalization
  • Model-based (physical model of illumination)

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Histogram Equalization

https://docs.opencv.org/3.4/d4/d1b/tutorial_histogram_equalization.html

• It is a method that improves the contrast in an image, in order to stretch out the intensity range.
• From the image below, you can see that the pixels seem clustered around the middle of the available range of intensities.

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

https://github.com/YuAo/Accelerated-CLAHE

• Histogram equalization (HE) is a method in image processing of contrast adjustment using the image's histogram.

• This method usually increases the global contrast of many images, especially when the usable data of the image is represented by close contrast values.

• Through this adjustment, the intensities can be better distributed on the histogram.

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

• Histogram Equalization stretch out this range.
• Equalization implies mapping one distribution (the given histogram) to another distribution (a wider and more uniform distribution of intensity values) so the intensity values are spread over the whole range.
• To accomplish the equalization effect, the remapping should be the cumulative distribution function (cdf). For the histogram H(i), its cumulative distribution H_cd(i) is (N: Number of pixels):

$${H}_{{{c}{d}}}{\left({i}\right)}=\frac{{\sum_{{{0}\le{j}<{i}}}{H}{\left({j}\right)}}}{{N}}$$

• Finally, we use a simple remapping procedure to obtain the intensity values of the equalized image:

$${I}_{{{e}{q}}}{\left({x},{y}\right)}={H}_{{{c}{d}}}{\left({I}{\left({x},{y}\right)}\right)}$$

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Cummulative Distribution Function (CDF)

use math,plot
m=matrix(runif(100),10,10)
h=hist(m,ylim=[0,1],breaks=20,plot=FALSE)
print(h$density)
cdf=vector('numeric',length(h$density))
for (i in 1:length(h$density)) {
  cdf[i]=sum(h$density[1:i])
}
plot(cdf,auto.scale=TRUE,main='CDF')

Higher order moment analysis of data series or vectors in R

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Spatial Image Intensity Equalization

• Intensity variations can be a result of a statistical process or due to the measuring technology and conditions

  • Variation can be considered as an overlay (addition) to the "real" measuring signal s(x,y)v(x,y)+n(x,y), and noise n

• Methods based on a spatial filtering of the images use the assumption that the bias field (intensity inhomogeneity) consists of a low spatial frequency intensity variation ⇒ Applying a High-pass filter in the wavenumber space!?

• Low-pass filtering methods can be used to extract non-uniformity

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Trivial Approach

• Assumption:
  1. There is only one axis in the image with low-frequency intensity variations due to inhomogeneous illumination
  2. The image content has statistically averaged homogeneous, i.e., equally distributed (small) features like cracks
• The mean image intensity I_mean(p) can be computed along a line l(p) (parametric equation, orientation by visual inspection along the strongest intensity variation/gradient) by using the average intensity along the perpendicular line at each point p:

$${x}_{{l}}={x}_{{0}}+{a}{p}\\ {y}_{{l}}={y}_{{0}}+{b}{p}\\ {l}{\left({p}\right)}:{p}\to{\left({x},{y}\right)}\\ {l}_{{\bot}}{\left({p},{q}\right)}:{q}\to{\left({x},{y}\right)}$$

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

(Left) Computing the average intensity I_avg(p) perpendicular to a line along the intensity gradient (Right) Correct all pixels perpendicular to the correction line with a equalization factor

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Contrast Limited Adaptive Histogram Equalization

https://github.com/YuAo/Accelerated-CLAHE

CLAHE (Contrast Limited Adaptive Histogram Equalization) is an algorithm for enhancing local contrast in images, and is frequently used in application areas like underwater photography, traffic control, astronomy, and medical imaging.

CLAHE can also be used in the tone mapping operation of displaying a HDR (High Dynamic Range) image.

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

• Adaptive histogram equalization (AHE) differs from ordinary histogram equalization in the respect that the adaptive method computes several histograms, each corresponding to a distinct section of the image, and uses them to redistribute the lightness values of the image.

• It is therefore suitable for improving the local contrast and enhancing the definitions of edges in each region of an image.

• AHE has a tendency to overamplify noise in relatively homogeneous regions of an image.

  • A variant of adaptive histogram equalization called contrast limited adaptive histogram equalization (CLAHE) prevents this by limiting the amplification.

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Frequency Transformation

• Time-dependent signal s(t) can be transformed in the frequency space S(ω) by using a frequency transformation, e.g., Discrete Fourier Transformation (DFT):

$${\left|{D}{F}{T}{\left({s}\right)}\right|}:{s}{\left({t}\right)}\to{S}{\left(\omega\right)}\\ {D}{F}{T}{\left({\left\lbrace{x}_{{n}}\right\rbrace}\right)}:{\left\lbrace{x}_{{n}}\right\rbrace}\to{\left\lbrace{X}_{{k}}\right\rbrace}\\ {X}_{{k}}=\sum_{{{0}\le{n}<{N}}}{x}_{{n}}{e}^{{\frac{{-{2}{i}\pi}}{{N}}{k}}}\\ {X}_{{k}}=\sum_{{{0}\le{n}<{N}}}{x}_{{n}}{\left({\cos{{\left(\frac{{{2}\pi}}{{N}}{k}{n}\right)}}}-{i}{\sin{{\left(\frac{{{2}\pi}}{{N}}{k}{n}\right)}}}\right)}$$

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

• The DFT transforms a series of complex numbers {x_n} into a sequence of complex numbers {X_k}.

  • The transformation is reversible (as long as complex numbers, i.e., magnitude and phase, is preserved).

• Low-, High-, and Bandpassfiltering can be performed by applying a mask function to the frequency distribution {X_k} and transforming back into time-space (blending in frequency space)

TU Graz, IVU_frequency_2017

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

2D DFT

• Images can be transformed into the frequency space, too, called wavenumber space

• A two-dimensional (2D) DFT is used (output is a matrix, too)

$${I}{F}{\left({k},{l}\right)}=\sum_{{{0}\le{m}<{N}}}\sum_{{{0}\le{n}<{N}}}{I}{\left({m},{n}\right)}{e}^{{-{2}{i}\pi{\left({k}\frac{{m}}{{N}}+{l}\frac{{n}}{{N}}\right)}}}$$

TU Graz, IVU_frequency_2017

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

TU Graz, IVU_frequency_2017

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Wavelet Decomposition

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Wavelet Decomposition

Parida et al.,2017 Decomposition of an image 2-D discrete wavelet transform with filter banks (2-D DWT)

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Wavelet Decomposition

Bosse et al., doi:10.3390/computers10030034 Example of a DWT signal decomposition of a US time-dependent signal

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Examples of 2D wavelets (Left) Haar (Right) Max Hat https://www.section.io/engineering-education/wavelet-transform-analysis-of-images-using-waveletanalyzer-toolbox-in-matlab/

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Wavelet Decomposition

• Instead performing a 2-D wavelet convolution, we can apply the 1-D transformation to the rows and columns of images as separable 2-D transformations.

• In most applications where wavelets are used for image processing, this approach is more practical due to the low computational complexity of separable transformations.

• Each decomposition reduces the image size by a factor 2 in each dimension: DWT: M × M → M/2 × M/2;

• The DWT decomposition can be repeated by using the ouput of the previous level

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Wavelet Decomposition

https://www.section.io/engineering-education/wavelet-transform-analysis-of-images-using-waveletanalyzer-toolbox-in-matlab/

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Wavelet Decomposition and Reconstruction

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Image Gradient

The (intensity) gradient of an image is the vector ∇I(x,y). It is characterized by a magnitude m and a direction φ in the image:

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Image Laplacian

Another important image transformation is the Laplacian of an image with intensity I(x,y) that is defined by:

• Invariant to image rotations.
• The laplacian is often used in image enhancement to increase contour effects

• Higher sensitivity to noise than the gradient.

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Edge Detection

Two main strategies:

1. Gradient strategy: detection of the local extrema in the gradient direction.
2. Laplacian strategy: detection of zero-crossing.

• These strategies rely on the fact that edges correspond to 0-order discontinuities of the intensity function.

• The derivative computation requires a pre-filtering of the images.

  • For instance: linear filtering for zero mean noises (e.g. white Gaussian noise and Gaussian filter) and non-linear filtering for impulse noise (median filter).
• Since all edge detection results are easily affected by the noise in the image, it is essential to filter out the noise to prevent false detection caused by it. To smooth the image, a Gaussian filter kernel is convolved with the image.

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Edge Detection: Sobel Derivative Filter

The Sobel filter is a x- and y-sensitive gradient filter by using a convolution operation with two 3×3 kernels.. The x- and y-gradients are merged finally in one image.

use math,imager,plot
img.sobel <- sobelEdges(img,blur=2,gradient=TRUE)
print(summary(img.sobel))
plot(img.sobel,auto.scale=TRUE)

Sobel edge filter. The gaussian blurring is essential to reduce noise.

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Edge Detection: Canny Filter

The canny edge filter is a multi-stage algorithm. After denoising, intensity gradients of the image are computed ofr x- and y-direction, then a non-maximum suppression is applied, finally applying a hysteris threhold filtering.

use math,imager,plot
img.canny <- cannyEdges(img,t1=0,t2=50,blur=4)
print(summary(img.canny))
plot(img.canny,auto.scale=TRUE)

Canny edge filter. The gaussian blurring is essential to reduce noise. The edge detection thresholds t₁ and t₂ relate to the intensity gradient and must be set carefully. https://docs.opencv.org/4.x/da/d22/tutorial_py_canny.html

PD Stefan Bosse - AFEML - Module A: Signal Features - Image Features

Kernel-based Convolution Algorithms

PD Stefan Bosse - AFEML - Module A: Signal Features - Geometric Transformations

Geometric Transformations

Simple geometrical operations of entire image or parts of the image are:

Advanced geometrical operations of entire image:

PD Stefan Bosse - AFEML - Module A: Signal Features - Geometric Distortions

Geometric Distortions

Local geometric distortions caused by optical imaging (lense distortion) https://www.image-engineering.de/library/image-quality/factors/1062-distortion

PD Stefan Bosse - AFEML - Module A: Signal Features - Measurement error and confidence

Measurement error and confidence

Systematic deviation (systematic error)

• Deviation is caused by the sensor, environment, and sometimes physical processes
• E.g.: incorrect calibration, constantly existing faults such as friction
• Can only be eliminated by carefully examining the source of the error

Random deviation (Random or statistical error)

• Deviation is caused by unavoidable, irregular disturbances
• with repeated measurement, individual results differ from each other
• Individual results vary by an average value

PD Stefan Bosse - AFEML - Module A: Signal Features - Measurement error and confidence

Measurement error and confidence

Random error scattering

• Random errors affect the accuracy of a measurement (noise).

• Noise affects input and target feature computation (ML output)!

• If one repeats a measurement of a quantity X which is falsified by pure random errors, the frequency distribution of the measured values is S = {s₁, s₂,...,s_n} by a mean value $\bar S$ given by a Gaussian distribution (the number of measurements N must be large).

PD Stefan Bosse - AFEML - Module A: Signal Features - Measurement error and confidence

9

Frequency distribution according to Gauss of measured values centered around an average value

PD Stefan Bosse - AFEML - Module A: Signal Features - Examples: Statistical Analysis

Examples: Statistical Analysis

PD Stefan Bosse - AFEML - Module A: Signal Features - Summary

Summary

PD Stefan Bosse - AFEML - Module A: Signal Features - Summary

Summary