# Summary and function reference

Below, `[]`

in an argument list means an optional argument.

## Image loading and saving

```
using FileIO
img = load("myimage.png")
save("imagecopy.jpg", img)
```

Standard test images are available in the TestImages package:

```
using TestImages
img = testimage("mandrill")
```

## Image construction, conversion, and views

Any array can be treated as an Image. In graphical environments, only arrays with `Colorant`

element types (`Gray`

, `RGB`

, `ARGB`

, etc.) are automatically displayed as images.

`ImageCore.colorview`

— Function.`colorview(C, A)`

returns a view of the numeric array `A`

, interpreting successive elements of `A`

as if they were channels of Colorant `C`

. This is almost identical to `ColorView{C}(A)`

, except that if `A`

is a `ChannelView`

, it will simply return the parent of `A`

, or use `reinterpret`

when appropriate. Consequently, the output may not be a `ColorView`

array.

Of relevance for types like RGB and BGR, the elements of `A`

are interpreted in constructor-argument order, not memory order (see `reinterpret`

if you want to use memory order).

**Example**

```
A = rand(3, 10, 10)
img = colorview(RGB, A)
```

`colorview(C, gray1, gray2, ...) -> imgC`

Combine numeric/grayscale images `gray1`

, `gray2`

, etc., into the separate color channels of an array `imgC`

with element type `C<:Colorant`

.

As a convenience, the constant `zeroarray`

fills in an array of matched size with all zeros.

**Example**

`imgC = colorview(RGB, r, zeroarray, b)`

creates an image with `r`

in the red chanel, `b`

in the blue channel, and nothing in the green channel.

See also: `StackedView`

.

`ImageCore.ColorView`

— Type.`ColorView{C}(A)`

creates a "view" of the numeric array `A`

, interpreting the first dimension of `A`

as if were the channels of a Colorant `C`

. The first dimension must have the proper number of elements for the constructor of `C`

. For example, if `A`

is a 3-by-m-by-n N0f8 array, `ColorView{RGB}(A)`

will create an m-by-n array with element type `RGB{N0f8}`

. Color spaces with a single element (i.e., grayscale) do not "consume" the first dimension of `A`

.

Of relevance for types like RGB and BGR, the elements of `A`

are interpreted in constructor-argument order, not memory order (see `reinterpret`

if you want to use memory order).

The opposite transformation is implemented by `ChannelView`

. See also `colorview`

.

`ImageCore.channelview`

— Function.`channelview(A)`

returns a view of `A`

, splitting out (if necessary) the color channels of `A`

into a new first dimension. This is almost identical to `ChannelView(A)`

, except that if `A`

is a `ColorView`

, it will simply return the parent of `A`

, or will use `reinterpret`

when appropriate. Consequently, the output may not be a `ChannelView`

array.

Of relevance for types like RGB and BGR, the channels of the returned array will be in constructor-argument order, not memory order (see `reinterpret`

if you want to use memory order).

`ImageCore.ChannelView`

— Type.`ChannelView(A)`

creates a "view" of the Colorant array `A`

, splitting out (if necessary) the separate color channels of `eltype(A)`

into a new first dimension. For example, if `A`

is a m-by-n RGB{N0f8} array, `ChannelView(A)`

will return a 3-by-m-by-n N0f8 array. Color spaces with a single element (i.e., grayscale) do not add a new first dimension of `A`

.

Of relevance for types like RGB and BGR, the channels of the returned array will be in constructor-argument order, not memory order (see `reinterpret`

if you want to use memory order).

The opposite transformation is implemented by `ColorView`

. See also `channelview`

.

`ImageCore.normedview`

— Function.`normedview([T], img::AbstractArray{Unsigned})`

returns a "view" of `img`

where the values are interpreted in terms of `Normed`

number types. For example, if `img`

is an `Array{UInt8}`

, the view will act like an `Array{N0f8}`

. Supply `T`

if the element type of `img`

is `UInt16`

, to specify whether you want a `N6f10`

, `N4f12`

, `N2f14`

, or `N0f16`

result.

`ImageCore.rawview`

— Function.`rawview(img::AbstractArray{FixedPoint})`

returns a "view" of `img`

where the values are interpreted in terms of their raw underlying storage. For example, if `img`

is an `Array{N0f8}`

, the view will act like an `Array{UInt8}`

.

`ImageCore.permuteddimsview`

— Function.`permuteddimsview(A, perm)`

returns a "view" of `A`

with its dimensions permuted as specified by `perm`

. This is like `permutedims`

, except that it produces a view rather than a copy of `A`

; consequently, any manipulations you make to the output will be mirrored in `A`

. Compared to the copy, the view is much faster to create, but generally slower to use.

`ImageCore.StackedView`

— Type.`StackedView(B, C, ...) -> A`

Present arrays `B`

, `C`

, etc, as if they are separate channels along the first dimension of `A`

. In particular,

```
B == A[1,:,:...]
C == A[2,:,:...]
```

and so on. Combined with `colorview`

, this allows one to combine two or more grayscale images into a single color image.

See also: `colorview`

.

`PaddedViews.paddedviews`

— Function.`Aspad = paddedviews(fillvalue, A1, A2, ....)`

Pad the arrays `A1`

, `A2`

, ..., to a common size or set of axes, chosen as the span of axes enclosing all of the input arrays.

**Example:**

```
julia> a1 = reshape([1,2], 2, 1)
2×1 Array{Int64,2}:
1
2
julia> a2 = [1.0,2.0]'
1×2 Array{Float64,2}:
1.0 2.0
julia> a1p, a2p = paddedviews(0, a1, a2);
julia> a1p
2×2 PaddedViews.PaddedView{Int64,2,Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},Array{Int64,2}}:
1 0
2 0
julia> a2p
2×2 PaddedViews.PaddedView{Float64,2,Tuple{Base.OneTo{Int64},Base.OneTo{Int64}},Array{Float64,2}}:
1.0 2.0
0.0 0.0
```

Images with defined geometry and axis meaning can be constructed using the `AxisArrays`

package:

```
using AxisArrays
img = AxisArray(A, (:y, :x, :time), (0.25μm, 0.25μm, 0.125s)) # see Unitful.jl for units
```

Custom metadata can be added as follows:

`img = ImageMeta(A, date=now(), patientID=12345)`

Any of these operations may be composed together, e.g., if you have an `m×n×3 UInt8`

array, you can put it in canonical RGB format and add metadata:

`img = ImageMeta(colorview(RGB, normedview(permuteddimsview(A, (3,1,2)))), sample="control")`

## Traits

These functions are the preferred way to access certain types of "internal" data about an image. They can sometimes be useful in allowing you to write generic code.

`ImageCore.pixelspacing`

— Function.`pixelspacing(img) -> (sx, sy, ...)`

Return a tuple representing the separation between adjacent pixels along each axis of the image. Defaults to (1,1,...). Use ImagesAxes for images with anisotropic spacing or to encode the spacing using physical units.

`ImageCore.spacedirections`

— Function.`spacedirections(img) -> (axis1, axis2, ...)`

Return a tuple-of-tuples, each `axis[i]`

representing the displacement vector between adjacent pixels along spatial axis `i`

of the image array, relative to some external coordinate system ("physical coordinates").

By default this is computed from `pixelspacing`

, but you can set this manually using ImagesMeta.

`spacedirections(img)`

Using ImageMetadata, you can set this property manually. For example, you could indicate that a photograph was taken with the camera tilted 30-degree relative to vertical using

`img["spacedirections"] = ((0.866025,-0.5),(0.5,0.866025))`

If not specified, it will be computed from `pixelspacing(img)`

, placing the spacing along the "diagonal". If desired, you can set this property in terms of physical units, and each axis can have distinct units.

`ImageCore.sdims`

— Function.`sdims(img)`

Return the number of spatial dimensions in the image. Defaults to the same as `ndims`

, but with ImagesAxes you can specify that some axes correspond to other quantities (e.g., time) and thus not included by `sdims`

.

`ImageCore.coords_spatial`

— Function.coords_spatial(img)

Return a tuple listing the spatial dimensions of `img`

.

Note that a better strategy may be to use ImagesAxes and take slices along the time axis.

`ImageCore.size_spatial`

— Function.`size_spatial(img)`

Return a tuple listing the sizes of the spatial dimensions of the image. Defaults to the same as `size`

, but using ImagesAxes you can mark some axes as being non-spatial.

`ImageCore.indices_spatial`

— Function.`indices_spatial(img)`

Return a tuple with the indices of the spatial dimensions of the image. Defaults to the same as `indices`

, but using ImagesAxes you can mark some axes as being non-spatial.

`ImageCore.nimages`

— Function.`nimages(img)`

Return the number of time-points in the image array. Defaults to

Use ImagesAxes if you want to use an explicit time dimension.

`ImageCore.assert_timedim_last`

— Function.`assert_timedim_last(img)`

Throw an error if the image has a time dimension that is not the last dimension.

## Element transformation and intensity scaling

`ImageCore.clamp01`

— Function.`clamp01(x) -> y`

Produce a value `y`

that lies between 0 and 1, and equal to `x`

when `x`

is already in this range. Equivalent to `clamp(x, 0, 1)`

for numeric values. For colors, this function is applied to each color channel separately.

See also: `clamp01nan`

.

`ImageCore.clamp01nan`

— Function.`clamp01nan(x) -> y`

Similar to `clamp01`

, except that any `NaN`

values are changed to 0.

See also: `clamp01`

.

`ImageCore.scaleminmax`

— Function.```
scaleminmax(min, max) -> f
scaleminmax(T, min, max) -> f
```

Return a function `f`

which maps values less than or equal to `min`

to 0, values greater than or equal to `max`

to 1, and uses a linear scale in between. `min`

and `max`

should be real values.

Optionally specify the return type `T`

. If `T`

is a colorant (e.g., RGB), then scaling is applied to each color channel.

**Examples**

**Example 1**

```
julia> f = scaleminmax(-10, 10)
(::#9) (generic function with 1 method)
julia> f(10)
1.0
julia> f(-10)
0.0
julia> f(5)
0.75
```

**Example 2**

```
julia> c = RGB(255.0,128.0,0.0)
RGB{Float64}(255.0,128.0,0.0)
julia> f = scaleminmax(RGB, 0, 255)
(::#13) (generic function with 1 method)
julia> f(c)
RGB{Float64}(1.0,0.5019607843137255,0.0)
```

See also: `takemap`

.

`ImageCore.scalesigned`

— Function.`scalesigned(maxabs) -> f`

Return a function `f`

which scales values in the range `[-maxabs, maxabs]`

(clamping values that lie outside this range) to the range `[-1, 1]`

.

See also: `colorsigned`

.

`scalesigned(min, center, max) -> f`

Return a function `f`

which scales values in the range `[min, center]`

to `[-1,0]`

and `[center,max]`

to `[0,1]`

. Values smaller than `min`

/`max`

get clamped to `min`

/`max`

, respectively.

See also: `colorsigned`

.

`ImageCore.colorsigned`

— Function.```
colorsigned()
colorsigned(colorneg, colorpos) -> f
colorsigned(colorneg, colorcenter, colorpos) -> f
```

Define a function that maps negative values (in the range [-1,0]) to the linear colormap between `colorneg`

and `colorcenter`

, and positive values (in the range [0,1]) to the linear colormap between `colorcenter`

and `colorpos`

.

The default colors are:

`colorcenter`

: white`colorneg`

: green1`colorpos`

: magenta

See also: `scalesigned`

.

`ImageCore.takemap`

— Function.```
takemap(f, A) -> fnew
takemap(f, T, A) -> fnew
```

Given a value-mapping function `f`

and an array `A`

, return a "concrete" mapping function `fnew`

. When applied to elements of `A`

, `fnew`

should return valid values for storage or display, for example in the range from 0 to 1 (for grayscale) or valid colorants. `fnew`

may be adapted to the actual values present in `A`

, and may not produce valid values for any inputs not in `A`

.

Optionally one can specify the output type `T`

that `fnew`

should produce.

**Example:**

```
julia> A = [0, 1, 1000];
julia> f = takemap(scaleminmax, A)
(::#7) (generic function with 1 method)
julia> f.(A)
3-element Array{Float64,1}:
0.0
0.001
1.0
```

## Storage-type transformation

`ImageCore.float32`

— Function.`float32.(img)`

converts the raw storage type of `img`

to `Float32`

, without changing the color space.

`ImageCore.float64`

— Function.`float64.(img)`

converts the raw storage type of `img`

to `Float64`

, without changing the color space.

`ImageCore.n0f8`

— Function.`n0f8.(img)`

converts the raw storage type of `img`

to `N0f8`

, without changing the color space.

`ImageCore.n6f10`

— Function.`n6f10.(img)`

converts the raw storage type of `img`

to `N6f10`

, without changing the color space.

`ImageCore.n4f12`

— Function.`n4f12.(img)`

converts the raw storage type of `img`

to `N4f12`

, without changing the color space.

`ImageCore.n2f14`

— Function.`n2f14.(img)`

converts the raw storage type of `img`

to `N2f14`

, without changing the color space.

`ImageCore.n0f16`

— Function.`n0f16.(img)`

converts the raw storage type of `img`

to `N0f16`

, without changing the color space.

## Color conversion

`imgg = Gray.(img)`

calculates a grayscale representation of a color image using the Rec 601 luma.

`imghsv = HSV.(img)`

converts to an HSV representation of color information.

## Image algorithms

### Linear filtering

`ImageFiltering.imfilter`

— Function.```
imfilter([T], img, kernel, [border="replicate"], [alg]) --> imgfilt
imfilter([r], img, kernel, [border="replicate"], [alg]) --> imgfilt
imfilter(r, T, img, kernel, [border="replicate"], [alg]) --> imgfilt
```

Filter a one, two or multidimensional array `img`

with a `kernel`

by computing their correlation.

**Details**

The term *filtering* emerges in the context of a Fourier transformation of an image, which maps an image from its canonical spatial domain to its concomitant frequency domain. Manipulating an image in the frequency domain amounts to retaining or discarding particular frequency components—a process analogous to sifting or filtering [1]. Because the Fourier transform establishes a link between the spatial and frequency representation of an image, one can interpret various image manipulations in the spatial domain as filtering operations which accept or reject specific frequencies.

The phrase *spatial filtering* is often used to emphasise that an operation is, at least conceptually, devised in the context of the spatial domain of an image. One further distinguishes between linear and non-linear spatial filtering. A filter is called linear if the operation performed on the pixels is linear, and is labeled non-linear otherwise.

An image filter can be represented by a function

where $k_i \in \mathbb{N}$ (i = 1,2). It is common to define $k_1 = 2a+1$ and $k_2 = 2b + 1$, where $a$ and $b$ are integers, which ensures that the filter dimensions are of odd size. Typically, $k_1$ equals $k_2$ and so, dropping the subscripts, one speaks of a $k \times k$ filter. Since the domain of the filter represents a grid of spatial coordinates, the filter is often called a mask and is visualized as a grid. For example, a $3 \times 3$ mask can be potrayed as follows:

The values of $w(s,t)$ are referred to as *filter coefficients*.

**Discrete convolution versus correlation**

There are two fundamental and closely related operations that one regularly performs on an image with a filter. The operations are called discrete *correlation* and *convolution*.

The correlation operation, denoted by the symbol $\star$, is given in two dimensions by the expression

whereas the comparable convolution operation, denoted by the symbol $\ast$, is given in two dimensions by

Since a digital image is of finite extent, both of these operations are undefined at the borders of the image. In particular, for an image of size $M \times N$, the function $f(x \pm s, y \pm t)$ is only defined for $1 \le x \pm s \le N$ and $1 \le y \pm t \le M$. In practice one addresses this problem by artificially expanding the domain of the image. For example, one can pad the image with zeros. Other padding strategies are possible, and they are discussed in more detail in the *Options* section of this documentation.

**One-dimensional illustration**

The difference between correlation and convolution is best understood with recourse to a one-dimensional example adapted from [1]. Suppose that a filter $w:\{-1,0,1\}\rightarrow \mathbb{R}$ has coefficients

Consider a discrete unit impulse function $f: \{x \in \mathbb{Z} \mid 1 \le x \le 7 \} \rightarrow \{0,1\}$ that has been padded with zeros. The function can be visualised as an image

The correlation operation can be interpreted as sliding $w$ along the image and computing the sum of products at each location. For example,

yields the output $g: \{x \in \mathbb{Z} \mid 1 \le x \le 7 \} \rightarrow \mathbb{R}$, which when visualized as a digital image, is equal to

The interpretation of the convolution operation is analogous to correlation, except that the filter $w$ has been rotated by 180 degrees. In particular,

yields the output $h: \{x \in \mathbb{Z} \mid 1 \le x \le 7 \} \rightarrow \mathbb{R}$ equal to

Instead of rotating the filter mask, one could instead rotate $f$ and still obtained the same convolution result. In fact, the conventional notation for convolution indicates that $f$ is flipped and not $w$. If $w$ is symmetric, then convolution and correlation give the same outcome.

**Two-dimensional illustration**

For a two-dimensional example, suppose the filter $w:\{-1, 0 ,1\} \times \{-1,0,1\} \rightarrow \mathbb{R}$ has coefficients

and consider a two-dimensional discrete unit impulse function

that has been padded with zeros:

The correlation operation $w(x,y) \star f(x,y)$ yields the output

whereas the convolution operation $w(x,y) \ast f(x,y)$ produces

**Discrete convolution and correlation as matrix multiplication**

Discrete convolution and correlation operations can also be formulated as a matrix multiplication, where one of the inputs is converted to a Toeplitz matrix, and the other is represented as a column vector. For example, consider a function $f:\{x \in \mathbb{N} \mid 1 \le x \le M \} \rightarrow \mathbb{R}$ and a filter $w: \{s \in \mathbb{N} \mid -k_1 \le s \le k_1 \} \rightarrow \mathbb{R}$. Then the matrix multiplication

is equivalent to the convolution $w(s) \ast f(x)$ assuming that the border of $f(x)$ has been padded with zeros.

To represent multidimensional convolution as matrix multiplication one reshapes the multidimensional arrays into column vectors and proceeds in an analogous manner. Naturally, the result of the matrix multiplication will need to be reshaped into an appropriate multidimensional array.

**Options**

The following subsections describe valid options for the function arguments in more detail.

**Choices for r**

You can dispatch to different implementations by passing in a resource `r`

as defined by the ComputationalResources package. For example,

` imfilter(ArrayFireLibs(), img, kernel)`

would request that the computation be performed on the GPU using the ArrayFire libraries.

**Choices for T**

Optionally, you can control the element type of the output image by passing in a type `T`

as the first argument.

**Choices for img**

You can specify a one, two or multidimensional array defining your image.

**Choices for kernel**

The `kernel[0,0,..]`

parameter corresponds to the origin (zero displacement) of the kernel; you can use `centered`

to place the origin at the array center, or use the OffsetArrays package to set `kernel`

's indices manually. For example, to filter with a random *centered* 3x3 kernel, you could use either of the following:

```
kernel = centered(rand(3,3))
kernel = OffsetArray(rand(3,3), -1:1, -1:1)
```

The `kernel`

parameter can be specified as an array or as a "factored kernel", a tuple `(filt1, filt2, ...)`

of filters to apply along each axis of the image. In cases where you know your kernel is separable, this format can speed processing. Each of these should have the same dimensionality as the image itself, and be shaped in a manner that indicates the filtering axis, e.g., a 3x1 filter for filtering the first dimension and a 1x3 filter for filtering the second dimension. In two dimensions, any kernel passed as a single matrix is checked for separability; if you want to eliminate that check, pass the kernel as a single-element tuple, `(kernel,)`

.

**Choices for border**

At the image edge, `border`

is used to specify the padding which will be used to extrapolate the image beyond its original bounds. As an indicative example of each option the results of the padding are illustrated on an image consisting of a row of six pixels which are specified alphabetically: $\boxed{a \, b \, c \, d \, e \, f}$. We show the effects of padding only on the left and right border, but analogous consequences hold for the top and bottom border.

`"replicate"`

(default)

The border pixels extend beyond the image boundaries.

See also: `Pad`

, `padarray`

, `Inner`

, `NA`

and `NoPad`

`"circular"`

The border pixels wrap around. For instance, indexing beyond the left border returns values starting from the right border.

See also: `Pad`

, `padarray`

, `Inner`

, `NA`

and `NoPad`

`"symmetric"`

The border pixels reflect relative to a position between pixels. That is, the border pixel is omitted when mirroring.

See also: `Pad`

, `padarray`

, `Inner`

, `NA`

and `NoPad`

`"reflect"`

The border pixels reflect relative to the edge itself.

See also: `Pad`

, `padarray`

, `Inner`

, `NA`

and `NoPad`

`Fill(m)`

The border pixels are filled with a specified value $m$.

See also: `Pad`

, `padarray`

, `Inner`

, `NA`

and `NoPad`

`Inner()`

Indicate that edges are to be discarded in filtering, only the interior of the result is to be returned.

See also: `Pad`

, `padarray`

, `Inner`

, `NA`

and `NoPad`

`NA()`

Choose filtering using "NA" (Not Available) boundary conditions. This is most appropriate for filters that have only positive weights, such as blurring filters.

See also: `Pad`

, `padarray`

, `Inner`

, `NA`

and `NoPad`

**Choices for alg**

The `alg`

parameter allows you to choose the particular algorithm: `FIR()`

(finite impulse response, aka traditional digital filtering) or `FFT()`

(Fourier-based filtering). If no choice is specified, one will be chosen based on the size of the image and kernel in a way that strives to deliver good performance. Alternatively you can use a custom filter type, like `KernelFactors.IIRGaussian`

.

**Examples**

The following subsections highlight some common use cases.

**Convolution versus correlation**

```
# Create a two-dimensional discrete unit impulse function.
f = fill(0,(9,9));
f[5,5] = 1;
# Specify a filter coefficient mask and set the center of the mask as the origin.
w = centered([1 2 3; 4 5 6 ; 7 8 9]);
#=
The default operation of `imfilter` is correlation. By reflecting `w` we
compute the convolution of `f` and `w`. `Fill(0,w)` indicates that we wish to
pad the border of `f` with zeros. The amount of padding is automatically
determined by considering the length of w.
=#
correlation = imfilter(f,w,Fill(0,w))
convolution = imfilter(f,reflect(w),Fill(0,w))
```

**Miscellaneous border padding options**

```
# Example function values f, and filter coefficients w.
f = reshape(1.0:81.0,9,9)
w = centered(reshape(1.0:9.0,3,3))
# You can designate the type of padding by specifying an appropriate string.
imfilter(f,w,"replicate")
imfilter(f,w,"replicate")
imfilter(f,w,"circular")
imfilter(f,w,"symmetric")
imfilter(f,w,"reflect")
# Alternatively, you can explicitly use the Pad type to designate the padding style.
imfilter(f,w,Pad(:replicate))
imfilter(f,w,Pad(:circular))
imfilter(f,w,Pad(:symmetric))
imfilter(f,w,Pad(:reflect))
# If you want to pad with a specific value then use the Fill type.
imfilter(f,w,Fill(0,w))
imfilter(f,w,Fill(1,w))
imfilter(f,w,Fill(-1,w))
#=
Specify 'Inner()' if you want to retrieve the interior sub-array of f for which
the filtering operation is defined without padding.
=#
imfilter(f,w,Inner())
```

**References**

R. C. Gonzalez and R. E. Woods.

*Digital Image Processing (3rd Edition)*. Upper Saddle River, NJ, USA: Prentice-Hall, 2006.

See also: `imfilter!`

, `centered`

, `padarray`

, `Pad`

, `Fill`

, `Inner`

, `KernelFactors.IIRGaussian`

.

`ImageFiltering.imfilter!`

— Function.```
imfilter!(imgfilt, img, kernel, [border="replicate"], [alg])
imfilter!(r, imgfilt, img, kernel, border, [inds])
imfilter!(r, imgfilt, img, kernel, border::NoPad, [inds=indices(imgfilt)])
```

Filter an array `img`

with kernel `kernel`

by computing their correlation, storing the result in `imgfilt`

.

The indices of `imgfilt`

determine the region over which the filtered image is computed–-you can use this fact to select just a specific region of interest, although be aware that the input `img`

might still get padded. Alteratively, explicitly provide the indices `inds`

of `imgfilt`

that you want to calculate, and use `NoPad`

boundary conditions. In such cases, you are responsible for supplying appropriate padding: `img`

must be indexable for all of the locations needed for calculating the output. This syntax is best-supported for FIR filtering; in particular, that that IIR filtering can lead to results that are inconsistent with respect to filtering the entire array.

See also: `imfilter`

.

`ImageFiltering.imgradients`

— Function.` imgradients(img, kernelfun=KernelFactors.ando3, border="replicate") -> gimg1, gimg2, ...`

Estimate the gradient of `img`

in the direction of the first and second dimension at all points of the image, using a kernel specified by `kernelfun`

.

**Output**

The gradient is returned as a tuple-of-arrays, one for each dimension of the input; `gimg1`

corresponds to the derivative with respect to the first dimension, `gimg2`

to the second, and so on.

**Details**

To appreciate the difference between various gradient estimation methods it is helpful to distinguish between: (1) a continuous scalar-valued *analogue* image $f_\textrm{A}(x_1,x_2)$, where $x_1,x_2 \in \mathbb{R}$, and (2) its discrete *digital* realization $f_\textrm{D}(x_1',x_2')$, where $x_1',x_2' \in \mathbb{N}$, $1 \le x_1' \le M$ and $1 \le x_2' \le N$.

**Analogue image**

The gradient of a continuous analogue image $f_{\textrm{A}}(x_1,x_2)$ at location $(x_1,x_2)$ is defined as the vector

where $\mathbf{e}_{d}$ $(d = 1,2)$ is the unit vector in the $x_d$-direction. The gradient points in the direction of maximum rate of change of $f_{\textrm{A}}$ at the coordinates $(x_1,x_2)$. The gradient can be used to compute the derivative of a function in an arbitrary direction. In particular, the derivative of $f_{\textrm{A}}$ in the direction of a unit vector $\mathbf{u}$ is given by $\nabla_{\mathbf{u}}f_\textrm{A}(x_1,x_2) = \nabla \mathbf{f}_{\textrm{A}}(x_1,x_2) \cdot \mathbf{u}$, where $\cdot$ denotes the dot product.

**Digital image**

In practice, we acquire a digital image $f_\textrm{D}(x_1',x_2')$ where the light intensity is known only at a discrete set of locations. This means that the required partial derivatives are undefined and need to be approximated using discrete difference formulae [1].

A straightforward way to approximate the partial derivatives is to use central-difference formulae

and

However, the central-difference formulae are very sensitive to noise. When working with noisy image data, one can obtain a better approximation of the partial derivatives by using a suitable weighted combination of the neighboring image intensities. The weighted combination can be represented as a *discrete convolution* operation between the image and a *kernel* which characterizes the requisite weights. In particular, if $h_{x_d}$ ($d = 1,2)$ represents a $2r+1 \times 2r+1$ kernel, then

The kernel is frequently also called a *mask* or *convolution matrix*.

**Weighting schemes and approximation error**

The choice of weights determines the magnitude of the approximation error and whether the finite-difference scheme is *isotropic*. A finite-difference scheme is isotropic if the approximation error does not depend on the orientation of the coordinate system and *anisotropic* if the approximation error has a directional bias [2]. With a continuous analogue image the magnitude of the gradient would be invariant upon rotation of the coordinate system, but in practice one cannot obtain perfect isotropy with a finite set of discrete points. Hence a finite-difference scheme is typically considered isotropic if the leading error term in the approximation does not have preferred directions.

Most finite-difference schemes that are used in image processing are based on $3 \times 3$ kernels, and as noted by [7], many can also be parametrized by a single parameter $\alpha$ as follows:

where

**Separable kernel**

A kernel is called *separable* if it can be expressed as the convolution of two one-dimensional filters. With a matrix representation of the kernel, separability means that the kernel matrix can be written as an outer product of two vectors. Separable kernels offer computational advantages since instead of performing a two-dimensional convolution one can perform a sequence of one-dimensional convolutions.

**Options**

You can specify your choice of the finite-difference scheme via the `kernelfun`

parameter. You can also indicate how to deal with the pixels on the border of the image with the `border`

parameter.

**Choices for kernelfun**

In general `kernelfun`

can be any function which satisfies the following interface:

` kernelfun(extended::NTuple{N,Bool}, d) -> kern_d,`

where `kern_d`

is the kernel for producing the derivative with respect to the $d$th dimension of an $N$-dimensional array. The parameter `extended[i]`

is true if the image is of size > 1 along dimension $i$. The parameter `kern_d`

may be provided as a dense or factored kernel, with factored representations recommended when the kernel is separable.

Some valid `kernelfun`

options are described below.

`KernelFactors.prewitt`

With the *prewit* option [3] the computation of the gradient is based on the kernels

See also: `KernelFactors.prewitt`

and `Kernel.prewitt`

`KernelFactors.sobel`

The *sobel* option [4] designates the kernels

See also: `KernelFactors.sobel`

and `Kernel.sobel`

`KernelFactors.ando3`

The *ando3* option [5] specifies the kernels

See also: `KernelFactors.ando3`

, and `Kernel.ando3`

; `KernelFactors.ando4`

, and `Kernel.ando4`

; `KernelFactors.ando5`

, and `Kernel.ando5`

`KernelFactors.scharr`

The *scharr* option [6] designates the kernels

See also: `KernelFactors.scharr`

and `Kernel.scharr`

`KernelFactors.bickley`

The *bickley* option [7,8] designates the kernels

See also: `KernelFactors.bickley`

and `Kernel.bickley`

**Choices for border**

At the image edge, `border`

is used to specify the padding which will be used to extrapolate the image beyond its original bounds. As an indicative example of each option the results of the padding are illustrated on an image consisting of a row of six pixels which are specified alphabetically: $\boxed{a \, b \, c \, d \, e \, f}$. We show the effects of padding only on the left and right border, but analogous consequences hold for the top and bottom border.

`"replicate"`

The border pixels extend beyond the image boundaries.

See also: `Pad`

, `padarray`

, `Inner`

and `NoPad`

`"circular"`

The border pixels wrap around. For instance, indexing beyond the left border returns values starting from the right border.

See also: `Pad`

, `padarray`

, `Inner`

and `NoPad`

`"symmetric"`

The border pixels reflect relative to a position between pixels. That is, the border pixel is omitted when mirroring.

See also: `Pad`

, `padarray`

, `Inner`

and `NoPad`

`"reflect"`

The border pixels reflect relative to the edge itself.

See also: `Pad`

, `padarray`

, `Inner`

and `NoPad`

**Example**

This example compares the quality of the gradient estimation methods in terms of the accuracy with which the orientation of the gradient is estimated.

```
using Images
values = linspace(-1,1,128);
w = 1.6*pi;
# Define a function of a sinusoidal grating, f(x,y) = sin( (w*x)^2 + (w*y)^2 ),
# together with its exact partial derivatives.
I = [sin( (w*x)^2 + (w*y)^2 ) for y in values, x in values];
Ix = [2*w*x*cos( (w*x)^2 + (w*y)^2 ) for y in values, x in values];
Iy = [2*w*y*cos( (w*x)^2 + (w*y)^2 ) for y in values, x in values];
# Determine the exact orientation of the gradients.
direction_true = atan.(Iy./Ix);
for kernelfunc in (KernelFactors.prewitt, KernelFactors.sobel,
KernelFactors.ando3, KernelFactors.scharr,
KernelFactors.bickley)
# Estimate the gradients and their orientations.
Gy, Gx = imgradients(I,kernelfunc, "replicate");
direction_estimated = atan.(Gy./Gx);
# Determine the mean absolute deviation between the estimated and true
# orientation. Ignore the values at the border since we expect them to be
# erroneous.
error = mean(abs.(direction_true[2:end-1,2:end-1] -
direction_estimated[2:end-1,2:end-1]));
error = round(error,5);
println("Using $kernelfunc results in a mean absolute deviation of $error")
end
# output
Using ImageFiltering.KernelFactors.prewitt results in a mean absolute deviation of 0.01069
Using ImageFiltering.KernelFactors.sobel results in a mean absolute deviation of 0.00522
Using ImageFiltering.KernelFactors.ando3 results in a mean absolute deviation of 0.00365
Using ImageFiltering.KernelFactors.scharr results in a mean absolute deviation of 0.00126
Using ImageFiltering.KernelFactors.bickley results in a mean absolute deviation of 0.00038
```

**References**

B. Jahne,

*Digital Image Processing*(5th ed.). Springer Publishing Company, Incorporated, 2005. 10.1007/3-540-27563-0M. Patra and M. Karttunen, "Stencils with isotropic discretization error for differential operators,"

*Numer. Methods Partial Differential Eq.*, vol. 22, pp. 936–953, 2006. doi:10.1002/num.20129J. M. Prewitt, "Object enhancement and extraction,"

*Picture processing and Psychopictorics*, vol. 10, no. 1, pp. 15–19, 1970.P.-E. Danielsson and O. Seger, "Generalized and separable sobel operators," in

*Machine Vision for Three-Dimensional Scenes*, H. Freeman, Ed. Academic Press, 1990, pp. 347–379. doi:10.1016/b978-0-12-266722-0.50016-6S. Ando, "Consistent gradient operators,"

*IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. doi:10.1109/34.841757H. Scharr and J. Weickert, "An anisotropic diffusion algorithm with optimized rotation invariance,"

*Mustererkennung 2000*, pp. 460–467, 2000. doi:10.1007/978-3-642-59802-9_58A. Belyaev, "Implicit image differentiation and filtering with applications to image sharpening,"

*SIAM Journal on Imaging Sciences*, vol. 6, no. 1, pp. 660–679, 2013. doi:10.1137/12087092xW. G. Bickley, "Finite difference formulae for the square lattice,"

*The Quarterly Journal of Mechanics and Applied Mathematics*, vol. 1, no. 1, pp. 35–42, 1948. doi:10.1093/qjmam/1.1.35

#### Kernel

`ImageFiltering.Kernel.sobel`

— Function.` diff1, diff2 = sobel()`

Return $3 \times 3$ kernels for two-dimensional gradient compution using the Sobel operator. The `diff1`

kernel computes the gradient along the y-axis (first dimension), and the `diff2`

kernel computes the gradient along the x-axis (second dimension).

**Citation**

P.-E. Danielsson and O. Seger, "Generalized and separable sobel operators," in *Machine Vision for Three-Dimensional Scenes*, H. Freeman, Ed. Academic Press, 1990, pp. 347–379. doi:10.1016/b978-0-12-266722-0.50016-6

See also: `KernelFactors.sobel`

, `Kernel.prewitt`

, `Kernel.ando3`

, `Kernel.scharr`

, `Kernel.bickley`

and `imgradients`

.

`ImageFiltering.Kernel.prewitt`

— Function.` diff1, diff2 = prewitt()`

Return $3 \times 3$ kernels for two-dimensional gradient compution using the Prewitt operator. The `diff1`

kernel computes the gradient along the y-axis (first dimension), and the `diff2`

kernel computes the gradient along the x-axis (second dimension).

**Citation**

J. M. Prewitt, "Object enhancement and extraction," *Picture processing and Psychopictorics*, vol. 10, no. 1, pp. 15–19, 1970.

See also: `KernelFactors.prewitt`

, `Kernel.sobel`

, `Kernel.ando3`

, `Kernel.scharr`

,`Kernel.bickley`

and `ImageFiltering.imgradients`

.

`ImageFiltering.Kernel.ando3`

— Function.` diff1, diff2 = ando3()`

Return $3 \times 3$ for two-dimensional gradient compution using Ando's "optimal" filters. The `diff1`

kernel computes the gradient along the y-axis (first dimension), and the `diff2`

kernel computes the gradient along the x-axis (second dimension).

**Citation**

S. Ando, "Consistent gradient operators," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. doi:10.1109/34.841757

See also: `KernelFactors.ando3`

, `Kernel.ando4`

, `Kernel.ando5`

and `ImageFiltering.imgradients`

.

`ImageFiltering.Kernel.ando4`

— Function.` diff1, diff2 = ando4()`

Return $4 \times 4$ kernels for two-dimensional gradient compution using Ando's "optimal" filters. The `diff1`

kernel computes the gradient along the y-axis (first dimension), and the `diff2`

kernel computes the gradient along the x-axis (second dimension).

**Citation**

S. Ando, "Consistent gradient operators," *IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. doi:10.1109/34.841757

See also: `KernelFactors.ando4`

, `Kernel.ando3`

, `Kernel.ando5`

and `ImageFiltering.imgradients`

.

`ImageFiltering.Kernel.ando5`

— Function.` diff1, diff2 = ando5()`

Return $5 \times 5$ kernels for two-dimensional gradient compution using Ando's "optimal" filters. The `diff1`

kernel computes the gradient along the y-axis (first dimension), and the `diff2`

kernel computes the gradient along the x-axis (second dimension).

**Citation**

*IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. doi:10.1109/34.841757

See also: `KernelFactors.ando5`

, `Kernel.ando3`

, `Kernel.ando4`

and `ImageFiltering.imgradients`

.

`ImageFiltering.Kernel.gaussian`

— Function.```
gaussian((σ1, σ2, ...), [(l1, l2, ...]) -> g
gaussian(σ) -> g
```

Construct a multidimensional gaussian filter, with standard deviation `σd`

along dimension `d`

. Optionally provide the kernel length `l`

, which must be a tuple of the same length.

If `σ`

is supplied as a single number, a symmetric 2d kernel is constructed.

See also: `KernelFactors.gaussian`

.

`ImageFiltering.Kernel.DoG`

— Function.```
DoG((σp1, σp2, ...), (σm1, σm2, ...), [l1, l2, ...]) -> k
DoG((σ1, σ2, ...)) -> k
DoG(σ::Real) -> k
```

Construct a multidimensional difference-of-gaussian kernel `k`

, equal to `gaussian(σp, l)-gaussian(σm, l)`

. When only a single `σ`

is supplied, the default is to choose `σp = σ, σm = √2 σ`

. Optionally provide the kernel length `l`

; the default is to extend by two `max(σp,σm)`

in each direction from the center. `l`

must be odd.

If `σ`

is provided as a single number, a symmetric 2d DoG kernel is returned.

See also: `KernelFactors.IIRGaussian`

.

`ImageFiltering.Kernel.LoG`

— Function.```
LoG((σ1, σ2, ...)) -> k
LoG(σ) -> k
```

Construct a Laplacian-of-Gaussian kernel `k`

. `σd`

is the gaussian width along dimension `d`

. If `σ`

is supplied as a single number, a symmetric 2d kernel is returned.

See also: `KernelFactors.IIRGaussian`

and `Kernel.Laplacian`

.

`ImageFiltering.Kernel.Laplacian`

— Type.```
Laplacian((true,true,false,...))
Laplacian(dims, N)
Laplacian()
```

Laplacian kernel in `N`

dimensions, taking derivatives along the directions marked as `true`

in the supplied tuple. Alternatively, one can pass `dims`

, a listing of the dimensions for differentiation. (However, this variant is not inferrable.)

`Laplacian()`

is the 2d laplacian, equivalent to `Laplacian((true,true))`

.

The kernel is represented as an opaque type, but you can use `convert(AbstractArray, L)`

to convert it into array format.

#### KernelFactors

`ImageFiltering.KernelFactors.sobel`

— Function.` kern1, kern2 = sobel()`

Return factored Sobel filters for dimensions 1 and 2 of a two-dimensional image. Each is a 2-tuple of one-dimensional filters.

**Citation**

P.-E. Danielsson and O. Seger, "Generalized and separable sobel operators," in *Machine Vision for Three-Dimensional Scenes*, H. Freeman, Ed. Academic Press, 1990, pp. 347–379. doi:10.1016/b978-0-12-266722-0.50016-6

See also: `Kernel.sobel`

and `ImageFiltering.imgradients`

.

` kern = sobel(extended::NTuple{N,Bool}, d)`

Return a factored Sobel filter for computing the gradient in `N`

dimensions along axis `d`

. If `extended[dim]`

is false, `kern`

will have size 1 along that dimension.

See also: `Kernel.sobel`

and `ImageFiltering.imgradients`

.

`ImageFiltering.KernelFactors.prewitt`

— Function.` kern1, kern2 = prewitt()`

Return factored Prewitt filters for dimensions 1 and 2 of your image. Each is a 2-tuple of one-dimensional filters.

**Citation**

J. M. Prewitt, "Object enhancement and extraction," *Picture processing and Psychopictorics*, vol. 10, no. 1, pp. 15–19, 1970.

See also: `Kernel.prewitt`

and `ImageFiltering.imgradients`

.

` kern = prewitt(extended::NTuple{N,Bool}, d)`

Return a factored Prewitt filter for computing the gradient in `N`

dimensions along axis `d`

. If `extended[dim]`

is false, `kern`

will have size 1 along that dimension.

See also: `Kernel.prewitt`

and `ImageFiltering.imgradients`

.

`ImageFiltering.KernelFactors.ando3`

— Function.` kern1, kern2 = ando3()`

Return a factored form of Ando's "optimal" $3 \times 3$ gradient filters for dimensions 1 and 2 of your image.

**Citation**

*IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. doi:10.1109/34.841757

See also: `Kernel.ando3`

,`KernelFactors.ando4`

, `KernelFactors.ando5`

and `ImageFiltering.imgradients`

.

` kern = ando3(extended::NTuple{N,Bool}, d)`

Return a factored Ando filter (size 3) for computing the gradient in `N`

dimensions along axis `d`

. If `extended[dim]`

is false, `kern`

will have size 1 along that dimension.

See also: `KernelFactors.ando4`

, `KernelFactors.ando5`

and `ImageFiltering.imgradients`

.

`ImageFiltering.KernelFactors.ando4`

— Function.` kern1, kern2 = ando4()`

Return separable approximations of Ando's "optimal" 4x4 filters for dimensions 1 and 2 of your image.

**Citation**

*IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. doi:10.1109/34.841757

See also: `Kernel.ando4`

and `ImageFiltering.imgradients`

.

` kern = ando4(extended::NTuple{N,Bool}, d)`

Return a factored Ando filter (size 4) for computing the gradient in `N`

dimensions along axis `d`

. If `extended[dim]`

is false, `kern`

will have size 1 along that dimension.

**Citation**

*IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. doi:10.1109/34.841757

See also: `Kernel.ando4`

and `ImageFiltering.imgradients`

.

`ImageFiltering.KernelFactors.ando5`

— Function.` kern1, kern2 = ando5()`

Return a separable approximations of Ando's "optimal" 5x5 gradient filters for dimensions 1 and 2 of your image.

**Citation**

*IEEE Transactions on Pattern Analysis and Machine Intelligence*, vol. 22, no.3, pp. 252–265, 2000. doi:10.1109/34.841757

See also: `Kernel.ando5`

and `ImageFiltering.imgradients`

.

` kern = ando5(extended::NTuple{N,Bool}, d)`

Return a factored Ando filter (size 5) for computing the gradient in `N`

dimensions along axis `d`

. If `extended[dim]`

is false, `kern`

will have size 1 along that dimension.

`ImageFiltering.KernelFactors.gaussian`

— Function.`gaussian(σ::Real, [l]) -> g`

Construct a 1d gaussian kernel `g`

with standard deviation `σ`

, optionally providing the kernel length `l`

. The default is to extend by two `σ`

in each direction from the center. `l`

must be odd.

`gaussian((σ1, σ2, ...), [l]) -> (g1, g2, ...)`

Construct a multidimensional gaussian filter as a product of single-dimension factors, with standard deviation `σd`

along dimension `d`

. Optionally provide the kernel length `l`

, which must be a tuple of the same length.

`ImageFiltering.KernelFactors.IIRGaussian`

— Function.`IIRGaussian([T], σ; emit_warning::Bool=true)`

Construct an infinite impulse response (IIR) approximation to a Gaussian of standard deviation `σ`

. `σ`

may either be a single real number or a tuple of numbers; in the latter case, a tuple of such filters will be created, each for filtering a different dimension of an array.

Optionally specify the type `T`

for the filter coefficients; if not supplied, it will match `σ`

(unless `σ`

is not floating-point, in which case `Float64`

will be chosen).

**Citation**

I. T. Young, L. J. van Vliet, and M. van Ginkel, "Recursive Gabor Filtering". IEEE Trans. Sig. Proc., 50: 2798-2805 (2002).

`TriggsSdika(a, b, scale, M)`

Defines a kernel for one-dimensional infinite impulse response (IIR) filtering. `a`

is a "forward" filter, `b`

a "backward" filter, `M`

is a matrix for matching boundary conditions at the right edge, and `scale`

is a constant scaling applied to each element at the conclusion of filtering.

**Citation**

B. Triggs and M. Sdika, "Boundary conditions for Young-van Vliet recursive filtering". IEEE Trans. on Sig. Proc. 54: 2365-2367 (2006).

`TriggsSdika(ab, scale)`

Create a symmetric Triggs-Sdika filter (with `a = b = ab`

). `M`

is calculated for you. Only length 3 filters are currently supported.

#### Kernel utilities

`ImageFiltering.centered`

— Function.`centered(kernel) -> shiftedkernel`

Shift the origin-of-coordinates to the center of `kernel`

. The center-element of `kernel`

will be accessed by `shiftedkernel[0, 0, ...]`

.

This function makes it easy to supply kernels using regular Arrays, and provides compatibility with other languages that do not support arbitrary indices.

See also: `imfilter`

.

`ImageFiltering.KernelFactors.kernelfactors`

— Function.`kernelfactors(factors::Tuple)`

Prepare a factored kernel for filtering. If passed a 2-tuple of vectors of lengths `m`

and `n`

, this will return a 2-tuple of `ReshapedVector`

s that are effectively of sizes `m×1`

and `1×n`

. In general, each successive `factor`

will be reshaped to extend along the corresponding dimension.

If passed a tuple of general arrays, it is assumed that each is shaped appropriately along its "leading" dimensions; the dimensionality of each is "extended" to `N = length(factors)`

, appending 1s to the size as needed.

`ImageFiltering.Kernel.reflect`

— Function.`reflect(kernel) --> reflectedkernel`

Compute the pointwise reflection around 0, 0, ... of the kernel `kernel`

. Using `imfilter`

with a `reflectedkernel`

performs convolution, rather than correlation, with respect to the original `kernel`

.

#### Boundaries and padding

`ImageFiltering.padarray`

— Function.` padarray([T], img, border) --> imgpadded`

Generate a padded image from an array `img`

and a specification `border`

of the boundary conditions and amount of padding to add.

**Output**

An expansion of the input image in which additional pixels are derived from the border of the input image using the extrapolation scheme specified by `border`

.

**Details**

The function supports one, two or multi-dimensional images. You can specify the element type `T`

of the output image.

**Options**

Valid `border`

options are described below.

`Pad`

The type `Pad`

designates the form of padding which should be used to extrapolate pixels beyond the boundary of an image. Instances must set `style`

, a Symbol specifying the boundary conditions of the image.

Symbol must be on one of:

`:replicate`

(repeat edge values to infinity),`:circular`

(image edges "wrap around"),`:symmetric`

(the image reflects relative to a position between pixels),`:reflect`

(the image reflects relative to the edge itself).

Refer to the documentation of `Pad`

for more details and examples for each option.

`Fill`

The type `Fill`

designates a particular value which will be used to extrapolate pixels beyond the boundary of an image. Refer to the documentation of `Fill`

for more details and illustrations.

**2D Examples**

Each example is based on the input array

**Examples with Pad**

The command `padarray(A, Pad(:replicate,4,4))`

yields

The command `padarray(A, Pad(:circular,4,4))`

yields

The command `padarray(A, Pad(:symmetric,4,4))`

yields

The command `padarray(A, Pad(:reflect,4,4))`

yields

**Examples with Fill**

The command `padarray(A, Fill(0,(4,4),(4,4)))`

yields

**3D Examples**

Each example is based on a multi-dimensional array $\mathsf{A} \in\mathbb{R}^{2 \times 2 \times 2}$ given by

Note that each example will yield a new multi-dimensional array $\mathsf{A}' \in \mathbb{R}^{4 \times 4 \times 4}$ of type `OffsetArray`

, where prepended dimensions may be negative or start from zero.

**Examples with Pad**

The command `padarray(A,Pad(:replicate,1,1,1))`

yields

The command `padarray(A,Pad(:circular,1,1,1))`

yields

The command `padarray(A,Pad(:symmetric,1,1,1))`

yields

The command `padarray(A,Pad(:reflect,1,1,1))`

yields

**Examples with Fill**

The command `padarray(A,Fill(0,(1,1,1)))`

yields

`ImageFiltering.Pad`

— Type.```
struct Pad{N} <: AbstractBorder
style::Symbol
lo::Dims{N} # number to extend by on the lower edge for each dimension
hi::Dims{N} # number to extend by on the upper edge for each dimension
end
```

`Pad`

is a type that designates the form of padding which should be used to extrapolate pixels beyond the boundary of an image. Instances must set `style`

, a Symbol specifying the boundary conditions of the image.

**Output**

The type `Pad`

specifying how the boundary of an image should be padded.

**Details**

When representing a spatial two-dimensional image filtering operation as a discrete convolution between the image and a $D \times D$ filter, the results are undefined for pixels closer than $D$ pixels from the border of the image. To define the operation near and at the border, one needs a scheme for extrapolating pixels beyond the edge. The `Pad`

type allows one to specify the necessary extrapolation scheme.

The type facilitates the padding of one, two or multi-dimensional images.

You can specify a different amount of padding at the lower and upper borders of each dimension of the image (top, left, bottom and right in two dimensions).

**Options**

Some valid `style`

options are described below. As an indicative example of each option the results of the padding are illustrated on an image consisting of a row of six pixels which are specified alphabetically: $\boxed{a \, b \, c \,d \, e \, f}$. We show the effects of padding only on the left and right border, but analogous consequences hold for the top and bottom border.

`:replicate`

(Default)

The border pixels extend beyond the image boundaries.

See also: `Fill`

, `padarray`

, `Inner`

and `NoPad`

`:circular`

The border pixels wrap around. For instance, indexing beyond the left border returns values starting from the right border.

See also: `Fill`

, `padarray`

, `Inner`

and `NoPad`

`:symmetric`

The border pixels reflect relative to a position between pixels. That is, the border pixel is omitted when mirroring.

See also: `Fill`

,`padarray`

, `Inner`

and `NoPad`

`:reflect`

The border pixels reflect relative to the edge itself.

See also: `Fill`

,`padarray`

, `Inner`

and `NoPad`

`ImageFiltering.Fill`

— Type.```
struct Fill{T,N} <: AbstractBorder
value::T
lo::Dims{N}
hi::Dims{N}
end
```

`Fill`

is a type that designates a particular value which will be used to extrapolate pixels beyond the boundary of an image.

**Output**

The type `Fill`

specifying the value with which the boundary of the image should be padded.

**Details**

When representing a two-dimensional spatial image filtering operation as a discrete convolution between an image and a $D \times D$ filter, the results are undefined for pixels closer than $D$ pixels from the border of the image. To define the operation near and at the border, one needs a scheme for extrapolating pixels beyond the edge. The `Fill`

type allows one to specify a particular value which will be used in the extrapolation. For more elaborate extrapolation schemes refer to the documentation of `Pad`

.

The type facilitates the padding of one, two or multi-dimensional images.

You can specify a different amount of padding at the lower and upper borders of each dimension of the image (top, left, bottom and right in two dimensions).

**Example**

As an indicative illustration consider an image consisting of a row of six pixels which are specified alphabetically: $\boxed{a \, b \, c \, d \, e \, f}$. We show the effects of padding with a constant value $m$ only on the left and right border, but analogous consequences hold for the top and bottom border.

See also: `Pad`

, `padarray`

, `Inner`

and `NoPad`

`ImageFiltering.Inner`

— Type.```
Inner()
Inner(lo, hi)
```

Indicate that edges are to be discarded in filtering, only the interior of the result is to be returned.

**Example:**

`imfilter(img, kernel, Inner())`

`ImageFiltering.NA`

— Type.```
NA()
NA(lo, hi)
```

Choose filtering using "NA" (Not Available) boundary conditions. This is most appropriate for filters that have only positive weights, such as blurring filters. Effectively, the output pixel value is normalized in the following way:

```
filtered img with Fill(0) boundary conditions
output = ---------------------------------------------
filtered 1 with Fill(0) boundary conditions
```

As a consequence, filtering has the same behavior as `nanmean`

. Indeed, invalid pixels in `img`

can be marked as `NaN`

and then they are effectively omitted from the filtered result.

`ImageFiltering.NoPad`

— Type.```
NoPad()
NoPad(border)
```

Indicates that no padding should be applied to the input array, or that you have already pre-padded the input image. Passing a `border`

object allows you to preserve "memory" of a border choice; it can be retrieved by indexing with `[]`

.

**Example**

The commands

```
np = NoPad(Pad(:replicate))
imfilter!(out, img, kernel, np)
```

run filtering directly, skipping any padding steps. Every entry of `out`

must be computable using in-bounds operations on `img`

and `kernel`

.

#### Algorithms

`ImageFiltering.Algorithm.FIR`

— Type.Filter using a direct algorithm

`ImageFiltering.Algorithm.FFT`

— Type.Filter using the Fast Fourier Transform

`ImageFiltering.Algorithm.IIR`

— Type.Filter with an Infinite Impulse Response filter

`ImageFiltering.Algorithm.Mixed`

— Type.Filter with a cascade of mixed types (IIR, FIR)

#### Internal machinery

`ReshapedOneD{N,Npre}(data)`

Return an object of dimensionality `N`

, where `data`

must have dimensionality 1. The indices are `0:0`

for the first `Npre`

dimensions, have the indices of `data`

for dimension `Npre+1`

, and are `0:0`

for the remaining dimensions.

`data`

must support `eltype`

and `ndims`

, but does not have to be an AbstractArray.

ReshapedOneDs allow one to specify a "filtering dimension" for a 1-dimensional filter.

### Nonlinear filtering and transformation

`ImageFiltering.MapWindow.mapwindow`

— Function.`mapwindow(f, img, window, [border="replicate"], [imginds=indices(img)]) -> imgf`

Apply `f`

to sliding windows of `img`

, with window size or indices specified by `window`

. For example, `mapwindow(median!, img, window)`

returns an `Array`

of values similar to `img`

(median-filtered, of course), whereas `mapwindow(extrema, img, window)`

returns an `Array`

of `(min,max)`

tuples over a window of size `window`

centered on each point of `img`

.

The function `f`

receives a buffer `buf`

for the window of data surrounding the current point. If `window`

is specified as a Dims-tuple (tuple-of-integers), then all the integers must be odd and the window is centered around the current image point. For example, if `window=(3,3)`

, then `f`

will receive an Array `buf`

corresponding to offsets `(-1:1, -1:1)`

from the `imgf[i,j]`

for which this is currently being computed. Alternatively, `window`

can be a tuple of AbstractUnitRanges, in which case the specified ranges are used for `buf`

; this allows you to use asymmetric windows if needed.

`border`

specifies how the edges of `img`

should be handled; see `imfilter`

for details.

Finally `imginds`

allows to omit unnecessary computations, if you want to do things like `mapwindow`

on a subimage, or a strided variant of mapwindow. It works as follows:

`mapwindow(f, img, window, border, (2:5, 1:2:7)) == mapwindow(f,img,window,border)[2:5, 1:2:7]`

Except more efficiently because it omits computation of the unused values.

For functions that can only take `AbstractVector`

inputs, you might have to first specialize `default_shape`

:

```
f = v->quantile(v, 0.75)
ImageFiltering.MapWindow.default_shape(::typeof(f)) = vec
```

and then `mapwindow(f, img, (m,n))`

should filter at the 75th quantile.

See also: `imfilter`

.

`Images.imROF`

— Function.`imgr = imROF(img, λ, iterations)`

Perform Rudin-Osher-Fatemi (ROF) filtering, more commonly known as Total Variation (TV) denoising or TV regularization. `λ`

is the regularization coefficient for the derivative, and `iterations`

is the number of relaxation iterations taken. 2d only.

See https://en.wikipedia.org/wiki/Total_variation_denoising and Chambolle, A. (2004). "An algorithm for total variation minimization and applications". Journal of Mathematical Imaging and Vision. 20: 89–97

### Edge detection

`Images.magnitude`

— Function.`m = magnitude(grad_x, grad_y)`

Calculates the magnitude of the gradient images given by `grad_x`

and `grad_y`

. Equivalent to $sqrt(grad_x.^2 + grad_y.^2)$.

Returns a magnitude image the same size as `grad_x`

and `grad_y`

.

`Images.phase`

— Function.`phase(grad_x, grad_y) -> p`

Calculate the rotation angle of the gradient given by `grad_x`

and `grad_y`

. Equivalent to `atan2(-grad_y, grad_x)`

, except that when both `grad_x`

and `grad_y`

are effectively zero, the corresponding angle is set to zero.

`Images.orientation`

— Function.`orientation(grad_x, grad_y) -> orient`

Calculate the orientation angle of the strongest edge from gradient images given by `grad_x`

and `grad_y`

. Equivalent to `atan2(grad_x, grad_y)`

. When both `grad_x`

and `grad_y`

are effectively zero, the corresponding angle is set to zero.

`Images.magnitude_phase`

— Function.`magnitude_phase(grad_x, grad_y) -> m, p`

Convenience function for calculating the magnitude and phase of the gradient images given in `grad_x`

and `grad_y`

. Returns a tuple containing the magnitude and phase images. See `magnitude`

and `phase`

for details.

`Images.imedge`

— Function.`grad_y, grad_x, mag, orient = imedge(img, kernelfun=KernelFactors.ando3, border="replicate")`

Edge-detection filtering. `kernelfun`

is a valid kernel function for `imgradients`

, defaulting to `KernelFactors.ando3`

. `border`

is any of the boundary conditions specified in `padarray`

.

Returns a tuple `(grad_y, grad_x, mag, orient)`

, which are the horizontal gradient, vertical gradient, and the magnitude and orientation of the strongest edge, respectively.

`Images.thin_edges`

— Function.```
thinned = thin_edges(img, gradientangle, [border])
thinned, subpix = thin_edges_subpix(img, gradientangle, [border])
thinned, subpix = thin_edges_nonmaxsup(img, gradientangle, [border]; [radius::Float64=1.35], [theta=pi/180])
thinned, subpix = thin_edges_nonmaxsup_subpix(img, gradientangle, [border]; [radius::Float64=1.35], [theta=pi/180])
```

Edge thinning for 2D edge images. Currently the only algorithm available is non-maximal suppression, which takes an edge image and its gradient angle, and checks each edge point for local maximality in the direction of the gradient. The returned image is non-zero only at maximal edge locations.

`border`

is any of the boundary conditions specified in `padarray`

.

In addition to the maximal edge image, the `_subpix`

versions of these functions also return an estimate of the subpixel location of each local maxima, as a 2D array or image of `Graphics.Point`

objects. Additionally, each local maxima is adjusted to the estimated value at the subpixel location.

Currently, the `_nonmaxsup`

functions are identical to the first two function calls, except that they also accept additional keyword arguments. `radius`

indicates the step size to use when searching in the direction of the gradient; values between 1.2 and 1.5 are suggested (default 1.35). `theta`

indicates the step size to use when discretizing angles in the `gradientangle`

image, in radians (default: 1 degree in radians = pi/180).

Example:

```
g = rgb2gray(rgb_image)
gx, gy = imgradients(g)
mag, grad_angle = magnitude_phase(gx,gy)
mag[mag .< 0.5] = 0.0 # Threshold magnitude image
thinned, subpix = thin_edges_subpix(mag, grad_angle)
```

`Images.canny`

— Function.`canny_edges = canny(img, (upper, lower), sigma=1.4)`

Performs Canny Edge Detection on the input image.

Parameters :

(upper, lower) : Bounds for hysteresis thresholding sigma : Specifies the standard deviation of the gaussian filter

**Example**

`imgedg = canny(img, (Percentile(80), Percentile(20)))`

### Corner Detection

`Images.imcorner`

— Function.```
corners = imcorner(img; [method])
corners = imcorner(img, threshold, percentile; [method])
```

Performs corner detection using one of the following methods -

```
1. harris
2. shi_tomasi
3. kitchen_rosenfeld
```

The parameters of the individual methods are described in their documentation. The maxima values of the resultant responses are taken as corners. If a threshold is specified, the values of the responses are thresholded to give the corner pixels. The threshold is assumed to be a percentile value unless `percentile`

is set to false.

`Images.harris`

— Function.`harris_response = harris(img; [k], [border], [weights])`

Performs Harris corner detection. The covariances can be taken using either a mean weighted filter or a gamma kernel.

`Images.shi_tomasi`

— Function.`shi_tomasi_response = shi_tomasi(img; [border], [weights])`

Performs Shi Tomasi corner detection. The covariances can be taken using either a mean weighted filter or a gamma kernel.

`Images.kitchen_rosenfeld`

— Function.`kitchen_rosenfeld_response = kitchen_rosenfeld(img; [border])`

Performs Kitchen Rosenfeld corner detection. The covariances can be taken using either a mean weighted filter or a gamma kernel.

`Images.fastcorners`

— Function.`fastcorners(img, n, threshold) -> corners`

Performs FAST Corner Detection. `n`

is the number of contiguous pixels which need to be greater (lesser) than intensity + threshold (intensity - threshold) for a pixel to be marked as a corner. The default value for n is 12.

### Feature Extraction

See the ImageFeatures package for a much more comprehensive set of tools.

`Images.blob_LoG`

— Function.`blob_LoG(img, σscales, [edges], [σshape]) -> Vector{BlobLoG}`

Find "blobs" in an N-D image using the negative Lapacian of Gaussians with the specifed vector or tuple of σ values. The algorithm searches for places where the filtered image (for a particular σ) is at a peak compared to all spatially- and σ-adjacent voxels, where σ is `σscales[i] * σshape`

for some i. By default, `σshape`

is an ntuple of 1s.

The optional `edges`

argument controls whether peaks on the edges are included. `edges`

can be `true`

or `false`

, or a N+1-tuple in which the first entry controls whether edge-σ values are eligible to serve as peaks, and the remaining N entries control each of the N dimensions of `img`

.

**Citation:**

Lindeberg T (1998), "Feature Detection with Automatic Scale Selection", International Journal of Computer Vision, 30(2), 79–116.

See also: `BlobLoG`

.

`Images.BlobLoG`

— Type.BlobLoG stores information about the location of peaks as discovered by `blob_LoG`

. It has fields:

location: the location of a peak in the filtered image (a CartesianIndex)

σ: the value of σ which lead to the largest

`-LoG`

-filtered amplitude at this locationamplitude: the value of the

`-LoG(σ)`

-filtered image at the peak

Note that the radius is equal to σ√2.

See also: `blob_LoG`

.

`Images.findlocalmaxima`

— Function.`findlocalmaxima(img, [region, edges]) -> Vector{CartesianIndex}`

Returns the coordinates of elements whose value is larger than all of their immediate neighbors. `region`

is a list of dimensions to consider. `edges`

is a boolean specifying whether to include the first and last elements of each dimension, or a tuple-of-Bool specifying edge behavior for each dimension separately.

`Images.findlocalminima`

— Function.Like `findlocalmaxima`

, but returns the coordinates of the smallest elements.

### Exposure

`Images.imhist`

— Function.```
edges, count = imhist(img, nbins)
edges, count = imhist(img, nbins, minval, maxval)
```

Generates a histogram for the image over nbins spread between `(minval, maxval]`

. If `minval`

and `maxval`

are not given, then the minimum and maximum values present in the image are taken.

`edges`

is a vector that specifies how the range is divided; `count[i+1]`

is the number of values `x`

that satisfy `edges[i] <= x < edges[i+1]`

. `count[1]`

is the number satisfying `x < edges[1]`

, and `count[end]`

is the number satisfying `x >= edges[end]`

. Consequently, `length(count) == length(edges)+1`

.

`Images.cliphist`

— Function.`clipped_hist = cliphist(hist, clip)`

Clips the histogram above a certain value `clip`

. The excess left in the bins exceeding `clip`

is redistributed among the remaining bins.

`Images.histeq`

— Function.```
hist_equalised_img = histeq(img, nbins)
hist_equalised_img = histeq(img, nbins, minval, maxval)
```

Returns a histogram equalised image with a granularity of approximately `nbins`

number of bins.

The `histeq`

function can handle a variety of input types. The returned image depends on the input type. If the input is an `Image`

then the resulting image is of the same type and has the same properties.

For coloured images, the input is converted to YIQ type and the Y channel is equalised. This is the combined with the I and Q channels and the resulting image converted to the same type as the input.

If minval and maxval are specified then intensities are equalized to the range (minval, maxval). The default values are 0 and 1.

`Images.adjust_gamma`

— Function.`gamma_corrected_img = adjust_gamma(img, gamma)`

Returns a gamma corrected image.

The `adjust_gamma`

function can handle a variety of input types. The returned image depends on the input type. If the input is an `Image`

then the resulting image is of the same type and has the same properties.

For coloured images, the input is converted to YIQ type and the Y channel is gamma corrected. This is the combined with the I and Q channels and the resulting image converted to the same type as the input.

`Images.imstretch`

— Function.`imgs = imstretch(img, m, slope)`

enhances or reduces (for slope > 1 or < 1, respectively) the contrast near saturation (0 and 1). This is essentially a symmetric gamma-correction. For a pixel of brightness `p`

, the new intensity is `1/(1+(m/(p+eps))^slope)`

.

This assumes the input `img`

has intensities between 0 and 1.

`Images.imadjustintensity`

— Function.`imadjustintensity(img [, (minval,maxval)]) -> Image`

Map intensities over the interval `(minval,maxval)`

to the interval `[0,1]`

. This is equivalent to `map(ScaleMinMax(eltype(img), minval, maxval), img)`

. (minval,maxval) defaults to `extrema(img)`

.

`Images.complement`

— Function.`y = complement(x)`

Take the complement `1-x`

of `x`

. If `x`

is a color with an alpha channel, the alpha channel is left untouched.

`Images.histmatch`

— Function.`hist_matched_img = histmatch(img, oimg, nbins)`

Returns a grayscale histogram matched image with a granularity of `nbins`

number of bins. `img`

is the image to be matched and `oimg`

is the image having the desired histogram to be matched to.

`Images.clahe`

— Function.```
hist_equalised_img = clahe(img, nbins, xblocks = 8, yblocks = 8, clip = 3)
```

Performs Contrast Limited Adaptive Histogram Equalisation (CLAHE) on the input image. It differs from ordinary histogram equalization in the respect that the adaptive method computes several histograms, each corresponding to a distinct section of the image, and uses them to redistribute the lightness values of the image. It is therefore suitable for improving the local contrast and enhancing the definitions of edges in each region of an image.

In the straightforward form, CLAHE is done by calculation a histogram of a window around each pixel and using the transformation function of the equalised histogram to rescale the pixel. Since this is computationally expensive, we use interpolation which gives a significant rise in efficiency without compromising the result. The image is divided into a grid and equalised histograms are calculated for each block. Then, each pixel is interpolated using the closest histograms.

The `xblocks`

and `yblocks`

specify the number of blocks to divide the input image into in each direction. `nbins`

specifies the granularity of histogram calculation of each local region. `clip`

specifies the value at which the histogram is clipped. The excess in the histogram bins with value exceeding `clip`

is redistributed among the other bins.

### Spatial transformations and resizing

`ImageTransformations.imresize`

— Function.```
imresize(img, sz) -> imgr
imresize(img, inds) -> imgr
```

Change `img`

to be of size `sz`

(or to have indices `inds`

). This interpolates the values at sub-pixel locations. If you are shrinking the image, you risk aliasing unless you low-pass filter `img`

first. For example:

```
σ = map((o,n)->0.75*o/n, size(img), sz)
kern = KernelFactors.gaussian(σ) # from ImageFiltering
imgr = imresize(imfilter(img, kern, NA()), sz)
```

See also `restrict`

.

`ImageTransformations.restrict`

— Function.`restrict(img[, region]) -> imgr`

Reduce the size of `img`

by two-fold along the dimensions listed in `region`

, or all spatial coordinates if `region`

is not specified. It anti-aliases the image as it goes, so is better than a naive summation over 2x2 blocks.

See also `imresize`

.

`ImageTransformations.warp`

— Function.`warp(img, tform, [indices], [degree = Linear()], [fill = NaN]) -> imgw`

Transform the coordinates of `img`

, returning a new `imgw`

satisfying `imgw[I] = img[tform(I)]`

. This approach is known as backward mode warping. The transformation `tform`

must accept a `SVector`

as input. A useful package to create a wide variety of such transformations is CoordinateTransformations.jl.

**Reconstruction scheme**

During warping, values for `img`

must be reconstructed at arbitrary locations `tform(I)`

which do not lie on to the lattice of pixels. How this reconstruction is done depends on the type of `img`

and the optional parameter `degree`

.

When `img`

is a plain array, then on-grid b-spline interpolation will be used. It is possible to configure what degree of b-spline to use with the parameter `degree`

. For example one can use `degree = Linear()`

for linear interpolation, `degree = Constant()`

for nearest neighbor interpolation, or `degree = Quadratic(Flat())`

for quadratic interpolation.

In the case `tform(I)`

maps to indices outside the original `img`

, those locations are set to a value `fill`

(which defaults to `NaN`

if the element type supports it, and `0`

otherwise). The parameter `fill`

also accepts extrapolation schemes, such as `Flat()`

, `Periodic()`

or `Reflect()`

.

For more control over the reconstruction scheme –- and how beyond-the-edge points are handled –- pass `img`

as an `AbstractInterpolation`

or `AbstractExtrapolation`

from Interpolations.jl.

**The meaning of the coordinates**

The output array `imgw`

has indices that would result from applying `inv(tform)`

to the indices of `img`

. This can be very handy for keeping track of how pixels in `imgw`

line up with pixels in `img`

.

If you just want a plain array, you can "strip" the custom indices with `parent(imgw)`

.

**Examples: a 2d rotation (see JuliaImages documentation for pictures)**

```
julia> using Images, CoordinateTransformations, TestImages, OffsetArrays
julia> img = testimage("lighthouse");
julia> indices(img)
(Base.OneTo(512),Base.OneTo(768))
# Rotate around the center of `img`
julia> tfm = recenter(RotMatrix(-pi/4), center(img))
AffineMap([0.707107 0.707107; -0.707107 0.707107], [-196.755,293.99])
julia> imgw = warp(img, tfm);
julia> indices(imgw)
(-196:709,-68:837)
# Alternatively, specify the origin in the image itself
julia> img0 = OffsetArray(img, -30:481, -384:383); # origin near top of image
julia> rot = LinearMap(RotMatrix(-pi/4))
LinearMap([0.707107 -0.707107; 0.707107 0.707107])
julia> imgw = warp(img0, rot);
julia> indices(imgw)
(-293:612,-293:611)
julia> imgr = parent(imgw);
julia> indices(imgr)
(Base.OneTo(906),Base.OneTo(905))
```

### Image statistics

`Images.minfinite`

— Function.`m = minfinite(A)`

calculates the minimum value in `A`

, ignoring any values that are not finite (Inf or NaN).

`Images.maxfinite`

— Function.`m = maxfinite(A)`

calculates the maximum value in `A`

, ignoring any values that are not finite (Inf or NaN).

`Images.maxabsfinite`

— Function.`m = maxabsfinite(A)`

calculates the maximum absolute value in `A`

, ignoring any values that are not finite (Inf or NaN).

`Images.meanfinite`

— Function.`M = meanfinite(img, region)`

calculates the mean value along the dimensions listed in `region`

, ignoring any non-finite values.

`Images.ssd`

— Function.`s = ssd(A, B)`

computes the sum-of-squared differences over arrays/images A and B

`Images.ssdn`

— Function.`s = ssdn(A, B)`

computes the sum-of-squared differences over arrays/images A and B, normalized by array size

`Images.sad`

— Function.`s = sad(A, B)`

computes the sum-of-absolute differences over arrays/images A and B

`Images.sadn`

— Function.`s = sadn(A, B)`

computes the sum-of-absolute differences over arrays/images A and B, normalized by array size

### Morphological operations

`ImageMorphology.dilate`

— Function.`imgd = dilate(img, [region])`

perform a max-filter over nearest-neighbors. The default is 8-connectivity in 2d, 27-connectivity in 3d, etc. You can specify the list of dimensions that you want to include in the connectivity, e.g., `region = [1,2]`

would exclude the third dimension from filtering.

`ImageMorphology.erode`

— Function.`imge = erode(img, [region])`

perform a min-filter over nearest-neighbors. The default is 8-connectivity in 2d, 27-connectivity in 3d, etc. You can specify the list of dimensions that you want to include in the connectivity, e.g., `region = [1,2]`

would exclude the third dimension from filtering.

`ImageMorphology.opening`

— Function.`imgo = opening(img, [region])`

performs the `opening`

morphology operation, equivalent to `dilate(erode(img))`

. `region`

allows you to control the dimensions over which this operation is performed.

`ImageMorphology.closing`

— Function.`imgc = closing(img, [region])`

performs the `closing`

morphology operation, equivalent to `erode(dilate(img))`

. `region`

allows you to control the dimensions over which this operation is performed.

`ImageMorphology.tophat`

— Function.`imgth = tophat(img, [region])`

performs `top hat`

of an image, which is defined as the image minus its morphological opening. `region`

allows you to control the dimensions over which this operation is performed.

`ImageMorphology.bothat`

— Function.`imgbh = bothat(img, [region])`

performs `bottom hat`

of an image, which is defined as its morphological closing minus the original image. `region`

allows you to control the dimensions over which this operation is performed.

`ImageMorphology.morphogradient`

— Function.`imgmg = morphogradient(img, [region])`

returns morphological gradient of the image, which is the difference between the dilation and the erosion of a given image. `region`

allows you to control the dimensions over which this operation is performed.

`ImageMorphology.morpholaplace`

— Function.`imgml = morpholaplace(img, [region])`

performs `Morphological Laplacian`

of an image, which is defined as the arithmetic difference between the internal and the external gradient. `region`

allows you to control the dimensions over which this operation is performed.

`Images.label_components`

— Function.```
label = label_components(tf, [connectivity])
label = label_components(tf, [region])
```

Find the connected components in a binary array `tf`

. There are two forms that `connectivity`

can take:

It can be a boolean array of the same dimensionality as

`tf`

, of size 1 or 3

along each dimension. Each entry in the array determines whether a given neighbor is used for connectivity analyses. For example, `connectivity = trues(3,3)`

would use 8-connectivity and test all pixels that touch the current one, even the corners.

You can provide a list indicating which dimensions are used to

determine connectivity. For example, `region = [1,3]`

would not test neighbors along dimension 2 for connectivity. This corresponds to just the nearest neighbors, i.e., 4-connectivity in 2d and 6-connectivity in 3d.

The default is `region = 1:ndims(A)`

.

The output `label`

is an integer array, where 0 is used for background pixels, and each connected region gets a different integer index.

`Images.component_boxes`

— Function.`component_boxes(labeled_array)`

-> an array of bounding boxes for each label, including the background label 0

`Images.component_lengths`

— Function.`component_lengths(labeled_array)`

-> an array of areas (2D), volumes (3D), etc. for each label, including the background label 0

`Images.component_indices`

— Function.`component_indices(labeled_array)`

-> an array of pixels for each label, including the background label 0

`Images.component_subscripts`

— Function.`component_subscripts(labeled_array)`

-> an array of pixels for each label, including the background label 0

`Images.component_centroids`

— Function.`component_centroids(labeled_array)`

-> an array of centroids for each label, including the background label 0

`Images.FeatureTransform.feature_transform`

— Function.`feature_transform(I::AbstractArray{Bool, N}, [w=nothing]) -> F`

Compute the feature transform of a binary image `I`

, finding the closest "feature" (positions where `I`

is `true`

) for each location in `I`

. Specifically, `F[i]`

is a `CartesianIndex`

encoding the position closest to `i`

for which `I[F[i]]`

is `true`

. In cases where two or more features in `I`

have the same distance from `i`

, an arbitrary feature is chosen. If `I`

has no `true`

values, then all locations are mapped to an index where each coordinate is `typemin(Int)`

.

Optionally specify the weight `w`

assigned to each coordinate. For example, if `I`

corresponds to an image where voxels are anisotropic, `w`

could be the voxel spacing along each coordinate axis. The default value of `nothing`

is equivalent to `w=(1,1,...)`

.

See also: `distance_transform`

.

**Citation**

'A Linear Time Algorithm for Computing Exact Euclidean Distance Transforms of Binary Images in Arbitrary Dimensions' Maurer et al., 2003

`Images.FeatureTransform.distance_transform`

— Function.`distance_transform(F::AbstractArray{CartesianIndex}, [w=nothing]) -> D`

Compute the distance transform of `F`

, where each element `F[i]`

represents a "target" or "feature" location assigned to `i`

. Specifically, `D[i]`

is the distance between `i`

and `F[i]`

. Optionally specify the weight `w`

assigned to each coordinate; the default value of `nothing`

is equivalent to `w=(1,1,...)`

.

See also: `feature_transform`

.

### Interpolation

`Images.bilinear_interpolation`

— Function.`P = bilinear_interpolation(img, r, c)`

Bilinear Interpolation is used to interpolate functions of two variables on a rectilinear 2D grid.

The interpolation is done in one direction first and then the values obtained are used to do the interpolation in the second direction.

### Integral Images

`Images.integral_image`

— Function.`integral_img = integral_image(img)`

Returns the integral image of an image. The integral image is calculated by assigning to each pixel the sum of all pixels above it and to its left, i.e. the rectangle from (1, 1) to the pixel. An integral image is a data structure which helps in efficient calculation of sum of pixels in a rectangular subset of an image. See `boxdiff`

for more information.

`Images.boxdiff`

— Function.```
sum = boxdiff(integral_image, ytop:ybot, xtop:xbot)
sum = boxdiff(integral_image, CartesianIndex(tl_y, tl_x), CartesianIndex(br_y, br_x))
sum = boxdiff(integral_image, tl_y, tl_x, br_y, br_x)
```

An integral image is a data structure which helps in efficient calculation of sum of pixels in a rectangular subset of an image. It stores at each pixel the sum of all pixels above it and to its left. The sum of a window in an image can be directly calculated using four array references of the integral image, irrespective of the size of the window, given the `yrange`

and `xrange`

of the window. Given an integral image -

```
A - - - - - - B -
- * * * * * * * -
- * * * * * * * -
- * * * * * * * -
- * * * * * * * -
- * * * * * * * -
C * * * * * * D -
- - - - - - - - -
```

The sum of pixels in the area denoted by * is given by S = D + A - B - C.

### Pyramids

`Images.gaussian_pyramid`

— Function.`pyramid = gaussian_pyramid(img, n_scales, downsample, sigma)`

Returns a gaussian pyramid of scales `n_scales`

, each downsampled by a factor `downsample`

and `sigma`

for the gaussian kernel.

### Phantoms

`Images.shepp_logan`

— Function.`phantom = shepp_logan(N,[M]; highContrast=true)`

output the NxM Shepp-Logan phantom, which is a standard test image usually used for comparing image reconstruction algorithms in the field of computed tomography (CT) and magnetic resonance imaging (MRI). If the argument M is omitted, the phantom is of size NxN. When setting the keyword argument `highConstrast`

to false, the CT version of the phantom is created. Otherwise, the high contrast MRI version is calculated.

## Image metadata utilities

`ImageMetadata.ImageMeta`

— Type.`ImageMeta`

is an AbstractArray that can have metadata, stored in a dictionary.

Construct an image with `ImageMeta(A, props)`

(for a properties dictionary `props`

), or with `Image(A, prop1=val1, prop2=val2, ...)`

.

`ImageAxes.data`

— Function.`data(img::ImageMeta) -> array`

Extract the data from `img`

, omitting the properties dictionary. `array`

shares storage with `img`

, so changes to one affect the other.

See also: `properties`

.

`ImageMetadata.properties`

— Function.`properties(imgmeta) -> props`

Extract the properties dictionary `props`

for `imgmeta`

. `props`

shares storage with `img`

, so changes to one affect the other.

See also: `data`

.

`ImageMetadata.copyproperties`

— Function.`copyproperties(img::ImageMeta, data) -> imgnew`

Create a new "image," copying the properties dictionary of `img`

but using the data of the AbstractArray `data`

. Note that changing the properties of `imgnew`

does not affect the properties of `img`

.

See also: `shareproperties`

.

`ImageMetadata.shareproperties`

— Function.`shareproperties(img::ImageMeta, data) -> imgnew`

Create a new "image," reusing the properties dictionary of `img`

but using the data of the AbstractArray `data`

. The two images have synchronized properties; modifying one also affects the other.

See also: `copyproperties`

.

`ImageMetadata.spatialproperties`

— Function.`spatialproperties(img)`

Return a vector of strings, containing the names of properties that have been declared "spatial" and hence should be permuted when calling `permutedims`

. Declare such properties like this:

`img["spatialproperties"] = ["spacedirections"]`

## Image segmentation

`ImageSegmentation.SegmentedImage`

— Type.`SegmentedImage`

type contains the index-label mapping, assigned labels, segment mean intensity and pixel count of each segment.

`ImageSegmentation.ImageEdge`

— Type.`edge = ImageEdge(index1, index2, weight)`

Construct an edge in a Region Adjacency Graph. `index1`

and `index2`

are the integers corresponding to individual pixels/voxels (in the sense of linear indexing via `sub2ind`

), and `weight`

is the edge weight (measures the dissimilarity between pixels/voxels).

`ImageSegmentation.labels_map`

— Function.`img_labeled = labels_map(seg)`

Return an array containing the label assigned to each pixel.

`ImageSegmentation.segment_labels`

— Function.`labels = segment_labels(seg)`

Returns the list of assigned labels

`ImageSegmentation.segment_pixel_count`

— Function.`c = segment_pixel_count(seg, l)`

Returns the count of pixels that are assigned label `l`

. If no label is supplied, it returns a Dict(label=>pixel_count) of all the labels.

`ImageSegmentation.segment_mean`

— Function.`m = segment_mean(seg, l)`

Returns the mean intensity of label `l`

. If no label is supplied, it returns a Dict(label=>mean) of all the labels.

`ImageSegmentation.seeded_region_growing`

— Function.```
seg_img = seeded_region_growing(img, seeds, [kernel_dim], [diff_fn])
seg_img = seeded_region_growing(img, seeds, [neighbourhood], [diff_fn])
```

Segments the N-D image `img`

using the seeded region growing algorithm and returns a `SegmentedImage`

containing information about the segments.

**Arguments:**

`img`

: N-D image to be segmented (arbitrary indices are allowed)`seeds`

:`Vector`

containing seeds. Each seed is a Tuple of a CartesianIndex{N} and a label. See below note for more information on labels.`kernel_dim`

: (Optional)`Vector{Int}`

having length N or a`NTuple{N,Int}`

whose ith element is an odd positive integer representing the length of the ith edge of the N-orthotopic neighbourhood`neighbourhood`

: (Optional) Function taking CartesianIndex{N} as input and returning the neighbourhood of that point.`diff_fn`

: (Optional) Function that returns a difference measure(δ) between the mean color of a region and color of a point

The labels attached to points must be positive integers, although multiple points can be assigned the same label. The output includes a labelled array that has same indexing as that of input image. Every index is assigned to either one of labels or a special label '0' indicating that the algorithm was unable to assign that index to a unique label.

**Examples**

```
julia> img = zeros(Gray{N0f8},4,4);
julia> img[2:4,2:4] = 1;
julia> seeds = [(CartesianIndex(3,1),1),(CartesianIndex(2,2),2)];
julia> seg = seeded_region_growing(img, seeds);
julia> labels_map(seg)
4×4 Array{Int64,2}:
1 1 1 1
1 2 2 2
1 2 2 2
1 2 2 2
```

**Citation:**

Albert Mehnert, Paul Jackaway (1997), "An improved seeded region growing algorithm", Pattern Recognition Letters 18 (1997), 1065-1071

`ImageSegmentation.unseeded_region_growing`

— Function.```
seg_img = unseeded_region_growing(img, threshold, [kernel_dim], [diff_fn])
seg_img = unseeded_region_growing(img, threshold, [neighbourhood], [diff_fn])
```

Segments the N-D image using automatic (unseeded) region growing algorithm and returns a `SegmentedImage`

containing information about the segments.

**Arguments:**

`img`

: N-D image to be segmented (arbitrary indices are allowed)`threshold`

: Upper bound of the difference measure (δ) for considering pixel into same segment`kernel_dim`

: (Optional)`Vector{Int}`

having length N or a`NTuple{N,Int}`

whose ith element is an odd positive integer representing the length of the ith edge of the N-orthotopic neighbourhood`neighbourhood`

: (Optional) Function taking CartesianIndex{N} as input and returning the neighbourhood of that point.`diff_fn`

: (Optional) Function that returns a difference measure (δ) between the mean color of a region and color of a point

**Examples**

```
julia> img = zeros(Gray{N0f8},4,4);
julia> img[2:4,2:4] = 1;
julia> seg = unseeded_region_growing(img, 0.2);
julia> labels_map(seg)
4×4 Array{Int64,2}:
1 1 1 1
1 2 2 2
1 2 2 2
1 2 2 2
```

`ImageSegmentation.felzenszwalb`

— Function.```
segments = felzenszwalb(img, k, [min_size])
index_map, num_segments = felzenszwalb(edges, num_vertices, k, [min_size])
```

Segments an image using Felzenszwalb's graph-based algorithm. The function can be used in either of two ways -

`segments = felzenszwalb(img, k, [min_size])`

Segments an image using Felzenszwalb's segmentation algorithm and returns the result as `SegmentedImage`

. The algorithm uses euclidean distance in color space as edge weights for the region adjacency graph.

Parameters:

img = input image

k = Threshold for region merging step. Larger threshold will result in bigger segments.

min_size = Minimum segment size

`index_map, num_segments = felzenszwalb(edges, num_vertices, k, [min_size])`

Segments an image represented as Region Adjacency Graph(RAG) using Felzenszwalb's segmentation algorithm. Each pixel/region corresponds to a node in the graph and weights on each edge measure the dissimilarity between pixels. The function returns the number of segments and index mapping from nodes of the RAG to segments.

Parameters:

edges = Array of edges in RAG. Each edge is represented as

`ImageEdge`

.num_vertices = Number of vertices in RAG

k = Threshold for region merging step. Larger threshold will result in bigger segments.

min_size = Minimum segment size

`ImageSegmentation.fast_scanning`

— Function.`seg_img = fast_scanning(img, threshold, [diff_fn])`

Segments the N-D image using a fast scanning algorithm and returns a `SegmentedImage`

containing information about the segments.

**Arguments:**

`img`

: N-D image to be segmented (arbitrary indices are allowed)`threshold`

: Upper bound of the difference measure (δ) for considering pixel into same segment; an`AbstractArray`

can be passed having same number of dimensions as that of`img`

for adaptive thresholding`diff_fn`

: (Optional) Function that returns a difference measure (δ) between the mean color of a region and color of a point

**Examples:**

```
julia> img = zeros(Float64, (3,3));
julia> img[2,:] = 0.5;
julia> img[:,2] = 0.6;
julia> seg = fast_scanning(img, 0.2);
julia> labels_map(seg)
3×3 Array{Int64,2}:
1 4 5
4 4 4
3 4 6
```

**Citation:**

Jian-Jiun Ding, Cheng-Jin Kuo, Wen-Chih Hong, "An efficient image segmentation technique by fast scanning and adaptive merging"

`ImageSegmentation.watershed`

— Function.`segments = watershed(img, markers)`

Segments the image using watershed transform. Each basin formed by watershed transform corresponds to a segment. If you are using image local minimas as markers, consider using hmin_transform to avoid oversegmentation.

Parameters:

img = input grayscale image

markers = An array (same size as img) with each region's marker assigned a index starting from 1. Zero means not a marker. If two markers have the same index, their regions will be merged into a single region. If you have markers as a boolean array, use

`label_components`

.

`ImageSegmentation.hmin_transform`

— Function.`out = hmin_transform(img, h)`

Suppresses all minima in grayscale image whose depth is less than h.

H-minima transform is defined as the reconstruction by erosion of (img + h) by img. See Morphological image analysis by Soille pg 170-172.

`ImageSegmentation.region_adjacency_graph`

— Function.`G, vert_map = region_adjacency_graph(seg, weight_fn)`

Constructs a region adjacency graph (RAG) from the `SegmentedImage`

. It returns the RAG along with a Dict(label=>vertex) map. `weight_fn`

is used to assign weights to the edges.

`weight_fn(label1, label2)`

Returns a real number corresponding to the weight of the edge between label1 and label2.

`ImageSegmentation.rem_segment`

— Function.`new_seg = rem_segment(seg, label, diff_fn)`

Removes the segment having label `label`

and returns the new `SegmentedImage`

. For more info, see `remove_segment!`

`ImageSegmentation.rem_segment!`

— Function.`rem_segment!(seg, label, diff_fn)`

In place removal of the segment having label `label`

, replacing it with the neighboring segment having least `diff_fn`

value.

`d = diff_fn(rem_label, neigh_label)`

A difference measure between label to be removed and its neighbors. `isless`

must be defined for objects of the type of `d`

.

**Examples**

```
# This removes the label `l` and replaces it with the label of
# neighbor having maximum pixel count.
julia> rem_segment!(seg, l, (i,j)->(-seg.segment_pixel_count[j]))
# This removes the label `l` and replaces it with the label of
# neighbor having the least value of euclidian metric.
julia> rem_segment!(seg, l, (i,j)->sum(abs2, seg.segment_means[i]-seg.segment_means[j]))
```

`ImageSegmentation.prune_segments`

— Function.`new_seg = prune_segments(seg, rem_labels, diff_fn)`

Removes all segments that have labels in `rem_labels`

replacing them with their neighbouring segment having least `diff_fn`

. `rem_labels`

is a `Vector`

of labels.

`new_seg = prune_segments(seg, is_rem, diff_fn)`

Removes all segments for which `is_rem`

returns true replacing them with their neighbouring segment having least `diff_fn`

.

`is_rem(label) -> Bool`

Returns true if label `label`

is to be removed otherwise false.

`d = diff_fn(rem_label, neigh_label)`

A difference measure between label to be removed and its neighbors. `isless`

must be defined for objects of the type of `d`

.

`ImageSegmentation.region_tree`

— Function.`t = region_tree(img, homogeneous)`

Creates a region tree from `img`

by splitting it recursively until all the regions are homogeneous.

`b = homogeneous(img)`

Returns true if `img`

is homogeneous.

**Examples**

```
# img is an array with elements of type `Float64`
julia> function homogeneous(img)
min, max = extrema(img)
max - min < 0.2
end
julia> t = region_tree(img, homogeneous);
```

`ImageSegmentation.region_splitting`

— Function.`seg = region_splitting(img, homogeneous)`

Segments `img`

by recursively splitting it until all the segments are homogeneous.

`b = homogeneous(img)`

Returns true if `img`

is homogeneous.

**Examples**

```
# img is an array with elements of type `Float64`
julia> function homogeneous(img)
min, max = extrema(img)
max - min < 0.2
end
julia> seg = region_splitting(img, homogeneous);
```

## ImageFeatures

### Types

`ImageFeatures.Feature`

— Type.`feature = Feature(keypoint, orientation = 0.0, scale = 0.0)`

The `Feature`

type has the keypoint, its orientation and its scale.

`ImageFeatures.Features`

— Type.```
features = Features(boolean_img)
features = Features(keypoints)
```

Returns a `Vector{Feature}`

of features generated from the `true`

values in a boolean image or from a list of keypoints.

`ImageFeatures.Keypoint`

— Type.```
keypoint = Keypoint(y, x)
keypoint = Keypoint(feature)
```

A `Keypoint`

may be created by passing the coordinates of the point or from a feature.

`ImageFeatures.Keypoints`

— Type.```
keypoints = Keypoints(boolean_img)
keypoints = Keypoints(features)
```

Creates a `Vector{Keypoint}`

of the `true`

values in a boolean image or from a list of features.

`ImageFeatures.BRIEF`

— Type.`brief_params = BRIEF([size = 128], [window = 9], [sigma = 2 ^ 0.5], [sampling_type = gaussian], [seed = 123])`

Argument | Type | Description |
---|---|---|

size | Int | Size of the descriptor |

window | Int | Size of sampling window |

sigma | Float64 | Value of sigma used for inital gaussian smoothing of image |

sampling_type | Function | Type of sampling used for building the descriptor (See BRIEF Sampling Patterns) |

seed | Int | Random seed used for generating the sampling pairs. For matching two descriptors, the seed used to build both should be same. |

`ImageFeatures.ORB`

— Type.`orb_params = ORB([num_keypoints = 500], [n_fast = 12], [threshold = 0.25], [harris_factor = 0.04], [downsample = 1.3], [levels = 8], [sigma = 1.2])`

Argument | Type | Description |
---|---|---|

num_keypoints | Int | Number of keypoints to extract and size of the descriptor calculated |

n_fast | Int | Number of consecutive pixels used for finding corners with FAST. See [`fastcorners` ] |

threshold | Float64 | Threshold used to find corners in FAST. See [`fastcorners` ] |

harris_factor | Float64 | Harris factor `k` used to rank keypoints by harris responses and extract the best ones |

downsample | Float64 | Downsampling parameter used while building the gaussian pyramid. See [`gaussian_pyramid` ] in Images.jl |

levels | Int | Number of levels in the gaussian pyramid. See [`gaussian_pyramid` ] in Images.jl |

sigma | Float64 | Used for gaussian smoothing in each level of the gaussian pyramid. See [`gaussian_pyramid` ] in Images.jl |

`ImageFeatures.FREAK`

— Type.`freak_params = FREAK([pattern_scale = 22.0])`

Argument | Type | Description |
---|---|---|

pattern_scale | Float64 | Scaling factor for the sampling window |

`ImageFeatures.BRISK`

— Type.`brisk_params = BRISK([pattern_scale = 1.0])`

Argument | Type | Description |
---|---|---|

`pattern_scale` | `Float64` | Scaling factor for the sampling window |

### Corners

`ImageFeatures.corner_orientations`

— Function.```
orientations = corner_orientations(img)
orientations = corner_orientations(img, corners)
orientations = corner_orientations(img, corners, kernel)
```

Returns the orientations of corner patches in an image. The orientation of a corner patch is denoted by the orientation of the vector between intensity centroid and the corner. The intensity centroid can be calculated as `C = (m01/m00, m10/m00)`

where mpq is defined as -

``mpq = (x^p)(y^q)I(y, x) for each p, q in the corner patch``

The kernel used for the patch can be given through the `kernel`

argument. The default kernel used is a gaussian kernel of size `5x5`

.

### BRIEF Sampling Patterns

`ImageFeatures.random_uniform`

— Function.`sample_one, sample_two = random_uniform(size, window, seed)`

Builds sampling pairs using random uniform sampling.

`ImageFeatures.random_coarse`

— Function.`sample_one, sample_two = random_coarse(size, window, seed)`

Builds sampling pairs using random sampling over a coarse grid.

`ImageFeatures.gaussian`

— Function.`sample_one, sample_two = gaussian(size, window, seed)`

Builds sampling pairs using gaussian sampling.

`ImageFeatures.gaussian_local`

— Function.`sample_one, sample_two = gaussian_local(size, window, seed)`

Pairs `(Xi, Yi)`

are randomly sampled using a Gaussian distribution where first `X`

is sampled with a standard deviation of `0.04*S^2`

and then the `Yi’s`

are sampled using a Gaussian distribution – Each `Yi`

is sampled with mean `Xi`

and standard deviation of `0.01 * S^2`

`ImageFeatures.center_sample`

— Function.`sample_one, sample_two = center_sample(size, window, seed)`

Builds sampling pairs `(Xi, Yi)`

where `Xi`

is `(0, 0)`

and `Yi`

is sampled uniformly from the window.

### Feature Description

`ImageFeatures.create_descriptor`

— Function.```
desc, keypoints = create_descriptor(img, keypoints, params)
desc, keypoints = create_descriptor(img, params)
```

Create a descriptor for each entry in `keypoints`

from the image `img`

. `params`

specifies the parameters for any of several descriptors:

Some descriptors support discovery of the `keypoints`

from `fastcorners`

.

### Feature Matching

`ImageFeatures.hamming_distance`

— Function.`distance = hamming_distance(desc_1, desc_2)`

Calculates the hamming distance between two descriptors.

`ImageFeatures.match_keypoints`

— Function.`matches = match_keypoints(keypoints_1, keypoints_2, desc_1, desc_2, threshold = 0.1)`

Finds matched keypoints using the `hamming_distance`

function having distance value less than `threshold`

.

### Texture Matching

#### Gray Level Co-occurence Matrix

```
glcm
glcm_symmetric
glcm_norm
glcm_prop
max_prob
contrast
ASM
IDM
glcm_entropy
energy
dissimilarity
correlation
glcm_mean_ref
glcm_mean_neighbour
glcm_var_ref
glcm_var_neighbour
```

#### Local Binary Patterns

```
lbp
modified_lbp
direction_coded_lbp
lbp_original
lbp_uniform
lbp_rotation_invariant
multi_block_lbp
```