Ctrl+K

Norms#

ivy.batch_norm(x, mean, variance, /, *, offset=None, scale=None, training=False, eps=1e-05, momentum=0.1, data_format='NSC', out=None)[source]#

Apply batch normalization to the input array and returns the normalized input, running mean and running variance arrays as output. If training == False, the mean and variance arrays passed as input are used for normalization and the same arrays are returned as running mean and running variance respectively. However, when training ==True, this function computes the batch mean and batch variance which is then used for normalization.In this case, the function returns the running mean and running variance calculated using the following formula:

running_mean = (1 - momentum) * running_mean + momentum * batch_mean running_var = (1 - momentum) * running_var + momentum * frac{n}{n-1} * batch_var

Parameters:

  • x (Union[NativeArray, Array]) –

    Input array of default shape (N, *S, C), where N is the batch dimension, *S corresponds to any number of spatial dimensions and

    C corresponds to the channel dimension.

  • mean (Optional[Union[Array, NativeArray]]) – Mean array used for input’s normalization. It can be of any shape braodcastable to (N,*S,C).

  • variance (Optional[Union[Array, NativeArray]]) – Variance array used for input’s normalization. It can be of any shape braodcastable to (N,*S,C).

  • offset (Optional[Union[Array, NativeArray]], default: None) – An offset array. If present, will be added to the normalized input. It can be of any shape broadcastable to (N,*S,C).

  • scale (Optional[Union[Array, NativeArray]], default: None) – A scale array. If present, the scale is applied to the normalized input. It can be of any shape broadcastable to (N,*S,C).

  • training (Optional[bool], default: False) – If true, calculate and use the mean and variance of x. Otherwise, use the provided mean and variance.

  • eps (Optional[float], default: 1e-05) – A small float number to avoid dividing by 0.

  • momentum (Optional[float], default: 0.1) –

    the value used for the running_mean and running_var computation.

    Default value is 0.1.

  • data_format (Optional[str], default: 'NSC') – The ordering of the dimensions in the input, one of “NSC” or “NCS”, where N is the batch dimension, S represents any number of spatial dimensions and C is the channel dimension. Default is “NSC”.

  • out (Optional[Tuple[Array, Array, Array]], default: None) – optional output arrays, for writing the result to.

Return type:

Tuple[Array, Array, Array]

Returns:

ret –

Tuple of arrays containing

the normalized input, running_mean, and running_variance.

ivy.group_norm(x, num_groups=1, /, *, offset=None, scale=None, eps=1e-05, data_format='NSC', out=None)[source]#

Apply group normalization to the input array and returns the normalized input.

Parameters:

  • x (Union[NativeArray, Array]) –

    Input array of default shape (N, *S, C), where N is the batch dimension, *S corresponds to any number of spatial dimensions and

    C corresponds to the channel dimension.

  • num_groups (int, default: 1) – number of groups to separate the channels into

  • offset (Optional[Union[Array, NativeArray]], default: None) – An offset array of size C. If present, will be added to the normalized input.

  • scale (Optional[Union[Array, NativeArray]], default: None) – A scale array of size C. If present, the scale is applied to the normalized input.

  • eps (Optional[float], default: 1e-05) – A small float number to avoid dividing by 0.

  • data_format (Optional[str], default: 'NSC') – The ordering of the dimensions in the input, one of “NSC” or “NCS”, where N is the batch dimension, S represents any number of spatial dimensions and C is the channel dimension. Default is “NSC”.

  • out (Optional[Array], default: None) – optional output arrays, for writing the result to.

Return type:

Array

Returns:

ret – The normalized array.

ivy.instance_norm(x, mean, variance, /, *, offset=None, scale=None, training=False, eps=0.0, momentum=0.1, data_format='NSC', out=None)[source]#

Apply instance normalization to the input array and returns the normalized input, running mean and running variance arrays as output. If training == False, the mean and variance arrays passed as input are used for normalization and the same arrays are returned as running mean and running variance respectively. However, when training ==True, this function computes the mean and variance across the spatial dimensions which is then used for normalization.In this case, the function returns the running mean and running variance calculated using the following formula:

running_mean = (1 - momentum) * running_mean + momentum * batch_mean running_var = (1 - momentum) * running_var + momentum * frac{n}{n-1} * batch_var

Parameters:

  • x (Union[NativeArray, Array]) –

    Input array of default shape (N, *S, C), where N is the batch dimension, *S corresponds to any number of spatial dimensions and

    C corresponds to the channel dimension.

  • mean (Optional[Union[Array, NativeArray]]) – Mean array of size C used for input’s normalization.

  • variance (Optional[Union[Array, NativeArray]]) – Variance array of size C used for input’s normalization.

  • offset (Optional[Union[Array, NativeArray]], default: None) – An offset array of size C. If present, will be added to the normalized input.

  • scale (Optional[Union[Array, NativeArray]], default: None) – A scale array of size C. If present, the scale is applied to the normalized input.

  • training (Optional[bool], default: False) – If true, calculate and use the mean and variance of x. Otherwise, use the provided mean and variance.

  • eps (Optional[float], default: 0.0) – A small float number to avoid dividing by 0.

  • momentum (Optional[float], default: 0.1) –

    the value used for the running_mean and running_var computation.

    Default value is 0.1.

  • data_format (Optional[str], default: 'NSC') – The ordering of the dimensions in the input, one of “NSC” or “NCS”, where N is the batch dimension, S represents any number of spatial dimensions and C is the channel dimension. Default is “NSC”.

  • out (Optional[Tuple[Array, Array, Array]], default: None) – optional output arrays, for writing the result to.

Return type:

Tuple[Array, Array, Array]

Returns:

ret –

Tuple of arrays containing

the normalized input, running_mean, and running_variance.

ivy.l1_normalize(x, /, *, axis=None, out=None)[source]#

Normalize the input array along the given axis to have L1 norm equal to 1.

Parameters:

  • x (Union[Array, NativeArray]) – Input array.

  • axis (Optional[Union[int, Tuple[int, ...]]], default: None) –

    Axis or axes along which to normalize. If None,

    the whole array is normalized.

  • out (Optional[Array], default: None) –

    Optional output array, for writing the result to.

    It must have a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret – The normalized array.

Examples

>>> x = ivy.array([[1., 2.], [3., 4.]])
>>> y = ivy.l1_normalize(x, axis=1)
>>> print(y)
ivy.array([[0.33333334, 1.33333337],
       [1.28571439, 2.28571439]])
ivy.l2_normalize(x, /, *, axis=None, out=None)[source]#

Normalize the input array along the given axis to have L2 norm equal to 1.

Parameters:

  • x (Union[Array, NativeArray]) – Input array.

  • axis (Optional[int], default: None) – Axis along which to normalize. If None, the whole array is normalized.

  • out (Optional[Array], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret – The normalized array.

Examples

>>> x = ivy.array([[1., 2.], [3., 4.]])
>>> y = ivy.l2_normalize(x, axis=1)
>>> print(y)
ivy.array([[0.44721359, 0.89442718],
       [0.60000002, 0.80000001]])
ivy.local_response_norm(x, size, /, *, bias=1.0, alpha=1.0, beta=0.5, average=False, data_format='NHWC', out=None)[source]#

Apply local response normalization across the channels of a 4D input array. The 4-D array is treated as a 3-D array of 1-D vectors (along the channel dimension), and each vector is normalized independently. Within a given vector, each component is divided by the squared sum of the neighbouring components.

Parameters:

  • x (Union[NativeArray, Array]) – Input array of default shape (N, H, W, C), where N is the batch dimension, H and W correspond to the spatial dimensions and C corresponds to the channel dimension.

  • size – The width of the normalization window.

  • alpha (Optional[float], default: 1.0) – The multiplicative factor.

  • beta (Optional[float], default: 0.5) – The exponent.

  • bias (Optional[float], default: 1.0) – An additive factor.

  • average (bool, default: False) – If True, each component is divided by the averaged squared sum.

  • data_format (Optional[Literal['NHWC', 'NCHW']], default: 'NHWC') – The ordering of the dimensions in the input, either “NHWC” or “NCHW”.

  • out (Optional[Tuple[Array, Array, Array]], default: None) – optional output arrays, for writing the result to.

Return type:

Array

Returns:

ret – The normalized array.

ivy.lp_normalize(x, /, *, p=2, axis=None, out=None)[source]#

Normalize the input array along the given axis to have Lp norm equal to 1.

Parameters:

  • x (Union[Array, NativeArray]) – Input array.

  • p (float, default: 2) – The Lp norm to use for normalization. Default is L2 norm (p=2).

  • axis (Optional[int], default: None) – Axis along which to normalize. If None, the whole array is normalized.

  • out (Optional[Array], default: None) – optional output array, for writing the result to. It must have a shape that the inputs broadcast to.

Return type:

Array

Returns:

ret – The normalized array.

Examples

>>> x = ivy.array([[1., 2.], [3., 4.]])
>>> y = ivy.lp_normalize(x, p=1, axis=1)
>>> print(y)
ivy.array([[0.33333334, 0.66666669],
       [0.42857143, 0.5714286 ]])

This should have hopefully given you an overview of the norms submodule, if you have any questions, please feel free to reach out on our discord!

On this page