Ctrl+K

Function testing#

ivy_tests.test_ivy.helpers.function_testing.args_to_container(array_args)[source]#
ivy_tests.test_ivy.helpers.function_testing.args_to_frontend(backend, *args, frontend_array_fn=None, include_derived=None, **kwargs)[source]#
ivy_tests.test_ivy.helpers.function_testing.arrays_to_frontend(backend, frontend_array_fn)[source]#
ivy_tests.test_ivy.helpers.function_testing.as_lists(*args)[source]#

Change the elements in args to be of type list.

ivy_tests.test_ivy.helpers.function_testing.convtrue(argument)[source]#

Convert NativeClass in argument to true framework counter part.

ivy_tests.test_ivy.helpers.function_testing.create_args_kwargs(*, backend, args_np, arg_np_vals, args_idxs, kwargs_np, kwarg_np_vals, kwargs_idxs, input_dtypes, test_flags, on_device)[source]#

Create arguments and keyword-arguments for the function to test.

Parameters:

  • args_np – A dictionary of arguments in Numpy.

  • kwargs_np – A dictionary of keyword-arguments in Numpy.

  • input_dtypes – data-types of the input arguments and keyword-arguments.

Returns:

Backend specific arguments, keyword-arguments

ivy_tests.test_ivy.helpers.function_testing.flatten(*, backend, ret)[source]#

Return a flattened numpy version of the arrays in ret.

ivy_tests.test_ivy.helpers.function_testing.flatten_and_to_np(*, backend, ret)[source]#
ivy_tests.test_ivy.helpers.function_testing.flatten_frontend(*, ret, backend, frontend_array_fn=None)[source]#

Return a flattened version of the frontend arrays in ret.

ivy_tests.test_ivy.helpers.function_testing.flatten_frontend_fw_to_np(frontend_ret, isscalar_func, is_native_array_func, to_numpy_func)[source]#
ivy_tests.test_ivy.helpers.function_testing.flatten_frontend_to_np(*, backend, ret)[source]#
ivy_tests.test_ivy.helpers.function_testing.get_frontend_ret(backend, frontend_fn, *args, frontend_array_function=None, precision_mode=False, test_trace=False, test_trace_each=False, **kwargs)[source]#
ivy_tests.test_ivy.helpers.function_testing.get_ret_and_flattened_np_array(backend_to_test, fn, *args, test_trace=False, test_trace_each=False, precision_mode=False, **kwargs)[source]#

Run func with args and kwargs.

Return the result along with its flattened version.

ivy_tests.test_ivy.helpers.function_testing.gradient_incompatible_function(*, fn)[source]#
ivy_tests.test_ivy.helpers.function_testing.gradient_test(*, fn, all_as_kwargs_np, args_np, kwargs_np, input_dtypes, test_flags, test_trace=False, test_trace_each=False, rtol_=None, atol_=1e-06, tolerance_dict=None, xs_grad_idxs=None, ret_grad_idxs=None, backend_to_test, ground_truth_backend, on_device)[source]#
ivy_tests.test_ivy.helpers.function_testing.gradient_unsupported_dtypes(*, fn)[source]#
ivy_tests.test_ivy.helpers.function_testing.kwargs_to_args_n_kwargs(*, num_positional_args, kwargs)[source]#

Split the kwargs into args and kwargs.

The first num_positional_args ported to args.

ivy_tests.test_ivy.helpers.function_testing.test_frontend_function(*, input_dtypes, test_flags, backend_to_test, on_device='cpu', frontend, fn_tree, gt_fn_tree=None, rtol=None, atol=1e-06, tolerance_dict=None, test_values=True, **all_as_kwargs_np)[source]#

Test a frontend function for the current backend by comparing the result with the function in the associated framework.

Parameters:

  • input_dtypes (Union[Dtype, List[Dtype]]) – data types of the input arguments in order.

  • test_flags (frontend_function_flags) – FunctionTestFlags object that stores all testing flags, including: num_positional_args, with_out, instance_method, as_variable, native_arrays, container, gradient, precision_mode

  • frontend (str) – current frontend (framework).

  • fn_tree (str) – Path to function in frontend framework namespace.

  • gt_fn_tree (Optional[str], default: None) – Path to function in ground truth framework namespace.

  • rtol (Optional[float], default: None) – relative tolerance value.

  • atol (float, default: 1e-06) – absolute tolerance value.

  • tolerance_dict (Optional[dict], default: None) – dictionary of tolerance values for specific dtypes.

  • test_values (bool, default: True) – if True, test for the correctness of the resulting values.

  • all_as_kwargs_np – input arguments to the function as keyword arguments. If an input argument has the same name as any other parameter of this function, add “arg_” before the argument name.

Returns:

  • ret – optional, return value from the function

  • ret_np – optional, return value from the Numpy function

ivy_tests.test_ivy.helpers.function_testing.test_frontend_method(*, init_input_dtypes=None, method_input_dtypes, init_flags, method_flags, init_all_as_kwargs_np=None, method_all_as_kwargs_np, frontend, frontend_method_data, backend_to_test, on_device, rtol_=None, atol_=1e-06, tolerance_dict=None, test_values=True)[source]#

Test a class-method that consumes (or returns) arrays for the current backend by comparing the result with numpy.

Parameters:

  • init_input_dtypes (Optional[Union[Dtype, List[Dtype]]], default: None) – data types of the input arguments to the constructor in order.

  • init_as_variable_flags – dictates whether the corresponding input argument passed to the constructor should be treated as an ivy.Variable.

  • init_num_positional_args – number of input arguments that must be passed as positional arguments to the constructor.

  • init_native_array_flags – dictates whether the corresponding input argument passed to the constructor should be treated as a native array.

  • init_all_as_kwargs_np (Optional[dict], default: None) – input arguments to the constructor as keyword arguments.

  • method_input_dtypes (Union[Dtype, List[Dtype]]) – data types of the input arguments to the method in order.

  • method_all_as_kwargs_np (dict) – input arguments to the method as keyword arguments.

  • frontend (str) – current frontend (framework).

  • rtol – relative tolerance value.

  • atol – absolute tolerance value.

  • tolerance_dict (Optional[dict], default: None) – dictionary of tolerance values for specific dtypes.

  • test_values (Union[bool, str], default: True) – can be a bool or a string to indicate whether correctness of values should be tested. If the value is with_v, shapes are tested but not values.

Returns:

  • ret – optional, return value from the function

  • ret_gt – optional, return value from the Ground Truth function

ivy_tests.test_ivy.helpers.function_testing.test_function(*, input_dtypes, test_flags, fn_name, rtol_=None, atol_=1e-06, tolerance_dict=None, test_values=True, xs_grad_idxs=None, ret_grad_idxs=None, backend_to_test, on_device, return_flat_np_arrays=False, **all_as_kwargs_np)[source]#

Test a function that consumes (or returns) arrays for the current backend by comparing the result with numpy.

Parameters:

  • input_dtypes (Union[Dtype, List[Dtype]]) – data types of the input arguments in order.

  • test_flags (FunctionTestFlags) – FunctionTestFlags object that stores all testing flags, including: num_positional_args, with_out, instance_method, as_variable, native_arrays, container, gradient

  • fw – current backend (framework).

  • fn_name (str) – name of the function to test.

  • rtol – relative tolerance value.

  • atol – absolute tolerance value.

  • test_values (bool, default: True) – if True, test for the correctness of the resulting values.

  • xs_grad_idxs (default: None) – Indices of the input arrays to compute gradients with respect to. If None, gradients are returned with respect to all input arrays. (Default value = None)

  • ret_grad_idxs (default: None) – Indices of the returned arrays for which to return computed gradients. If None, gradients are returned for all returned arrays. (Default value = None)

  • on_device (str) – The device on which to create arrays

  • return_flat_np_arrays (bool, default: False) – If test_values is False, this flag dictates whether the original returns are returned, or whether the flattened numpy arrays are returned.

  • all_as_kwargs_np – input arguments to the function as keyword arguments.

Returns:

  • ret – optional, return value from the function

  • ret_gt – optional, return value from the Ground Truth function

Examples

>>> input_dtypes = 'float64'
>>> as_variable_flags = False
>>> with_out = False
>>> num_positional_args = 0
>>> native_array_flags = False
>>> container_flags = False
>>> instance_method = False
>>> test_flags = FunctionTestFlags(num_positional_args, with_out,
    instance_method,
    as_variable,
    native_arrays,
    container_flags,
    none)
>>> fw = "torch"
>>> fn_name = "abs"
>>> x = np.array([-1])
>>> test_function(input_dtypes, test_flags, fw, fn_name, x=x)

>>> input_dtypes = ['float64', 'float32']
>>> as_variable_flags = [False, True]
>>> with_out = False
>>> num_positional_args = 1
>>> native_array_flags = [True, False]
>>> container_flags = [False, False]
>>> instance_method = False
>>> test_flags = FunctionTestFlags(num_positional_args, with_out,
    instance_method,
    as_variable,
    native_arrays,
    container_flags,
    none)
>>> fw = "numpy"
>>> fn_name = "add"
>>> x1 = np.array([1, 3, 4])
>>> x2 = np.array([-3, 15, 24])
>>> test_function(input_dtypes, test_flags, fw, fn_name, x1=x1, x2=x2)
ivy_tests.test_ivy.helpers.function_testing.test_function_backend_computation(fw, test_flags, all_as_kwargs_np, input_dtypes, on_device, fn_name)[source]#
ivy_tests.test_ivy.helpers.function_testing.test_function_ground_truth_computation(ground_truth_backend, on_device, args_np, arg_np_arrays, arrays_args_indices, kwargs_np, arrays_kwargs_indices, kwarg_np_arrays, input_dtypes, test_flags, fn_name)[source]#
ivy_tests.test_ivy.helpers.function_testing.test_gradient_backend_computation(backend_to_test, args_np, arg_np_vals, args_idxs, kwargs_np, kwarg_np_vals, kwargs_idxs, input_dtypes, test_flags, on_device, fn, test_trace, test_trace_each, xs_grad_idxs, ret_grad_idxs)[source]#
ivy_tests.test_ivy.helpers.function_testing.test_gradient_ground_truth_computation(ground_truth_backend, on_device, fn, input_dtypes, all_as_kwargs_np, args_np, arg_np_vals, args_idxs, kwargs_np, kwarg_np_vals, test_flags, kwargs_idxs, test_trace, test_trace_each, xs_grad_idxs, ret_grad_idxs)[source]#
ivy_tests.test_ivy.helpers.function_testing.test_method(*, init_input_dtypes=None, method_input_dtypes=None, init_all_as_kwargs_np=None, method_all_as_kwargs_np=None, init_flags, method_flags, class_name, method_name='__call__', init_with_v=False, method_with_v=False, rtol_=None, atol_=1e-06, tolerance_dict=None, test_values=True, test_gradients=False, xs_grad_idxs=None, ret_grad_idxs=None, test_trace=False, test_trace_each=False, backend_to_test, ground_truth_backend, on_device, return_flat_np_arrays=False)[source]#

Test a class-method that consumes (or returns) arrays for the current backend by comparing the result with numpy.

Parameters:

  • init_input_dtypes (Optional[List[Dtype]], default: None) – data types of the input arguments to the constructor in order.

  • init_as_variable_flags – dictates whether the corresponding input argument passed to the constructor should be treated as an ivy.Array.

  • init_num_positional_args – number of input arguments that must be passed as positional arguments to the constructor.

  • init_native_array_flags – dictates whether the corresponding input argument passed to the constructor should be treated as a native array.

  • init_all_as_kwargs_np (Optional[dict], default: None) – input arguments to the constructor as keyword arguments.

  • method_input_dtypes (Optional[List[Dtype]], default: None) – data types of the input arguments to the method in order.

  • method_as_variable_flags – dictates whether the corresponding input argument passed to the method should be treated as an ivy.Array.

  • method_num_positional_args – number of input arguments that must be passed as positional arguments to the method.

  • method_native_array_flags – dictates whether the corresponding input argument passed to the method should be treated as a native array.

  • method_container_flags – dictates whether the corresponding input argument passed to the method should be treated as an ivy Container.

  • method_all_as_kwargs_np (Optional[dict], default: None) – input arguments to the method as keyword arguments.

  • class_name (str) – name of the class to test.

  • method_name (str, default: '__call__') – name of the method to test.

  • init_with_v (bool, default: False) – if the class being tested is an ivy.Module, then setting this flag as True will call the constructor with the variables v passed explicitly.

  • method_with_v (bool, default: False) – if the class being tested is an ivy.Module, then setting this flag as True will call the method with the variables v passed explicitly.

  • rtol – relative tolerance value.

  • atol – absolute tolerance value.

  • test_values (Union[bool, str], default: True) – can be a bool or a string to indicate whether correctness of values should be tested. If the value is with_v, shapes are tested but not values.

  • test_gradients (bool, default: False) – if True, test for the correctness of gradients.

  • xs_grad_idxs (default: None) – Indices of the input arrays to compute gradients with respect to. If None, gradients are returned with respect to all input arrays. (Default value = None)

  • ret_grad_idxs (default: None) – Indices of the returned arrays for which to return computed gradients. If None, gradients are returned for all returned arrays. (Default value = None)

  • test_trace (bool, default: False) – If True, test for the correctness of tracing.

  • test_trace_each (bool, default: False) – If True, test for the correctness of tracing (tracing each example separately)

  • ground_truth_backend (str) – Ground Truth Backend to compare the result-values.

  • device – The device on which to create arrays.

  • return_flat_np_arrays (bool, default: False) – If test_values is False, this flag dictates whether the original returns are returned, or whether the flattened numpy arrays are returned.

Returns:

  • ret – optional, return value from the function

  • ret_gt – optional, return value from the Ground Truth function

ivy_tests.test_ivy.helpers.function_testing.test_method_backend_computation(init_input_dtypes, init_flags, backend_to_test, init_all_as_kwargs_np, on_device, method_input_dtypes, method_flags, method_all_as_kwargs_np, class_name, method_name, init_with_v, test_trace, test_trace_each, method_with_v)[source]#
ivy_tests.test_ivy.helpers.function_testing.test_method_ground_truth_computation(ground_truth_backend, on_device, org_con_data, args_np_method, met_arg_np_vals, met_args_idxs, kwargs_np_method, met_kwarg_np_vals, met_kwargs_idxs, method_input_dtypes, method_flags, class_name, method_name, test_trace, test_trace_each, v_np)[source]#
ivy_tests.test_ivy.helpers.function_testing.traced_if_required(backend, fn, test_trace=False, test_trace_each=False, args=None, kwargs=None)[source]#
ivy_tests.test_ivy.helpers.function_testing.wrap_frontend_function_args(argument)[source]#

Wrap frontend function arguments to return native arrays.

This should have hopefully given you an overview of the function_testing submodule, if you have any questions, please feel free to reach out on our discord!

On this page