## Numerical Stability

### NoNansAssertion

CLASS efemarai.assertions.NoNansAssertion(TensorAssertion)

Ensure that no tensor contains any `NaN` values.

Registered by default.

Examples:

```ef.register_assertion(NoNansAssertion())

ef.deregister_assertion(NoNansAssertion)```

### NoInfsAssertion

CLASS efemarai.assertions.NoInfsAssertion(TensorAssertion)

Ensure that no tensor contains any `Inf` values.

Registered by default.

Examples:

```ef.register_assertion(NoInfsAssertion())

ef.deregister_assertion(NoInfsAssertion)```

Ensure that all gradients contain only zeros before the backward pass is executed.

Registered by default.

Examples:

```ef.register_assertion(NoNonZeroGradientsAssertion())

### NoExplodingTensors

CLASS efemarai.assertions.NoExplodingTensors(FunctionAssertion)

Ensure that no tensor in the computational graph has an element less / greater than a certain threshold.

`def __init__(self, threshold=1e5)`

Initialize the assertion.

Parameters

• threshold (`float`) – if a tensor has elements with absolute value greater than this threshold then the tensor is considered ‘exploding’.

Examples:

`ef.register_assertion(NoExlodingTensors(threshold=1e2))`

### NoVanishingTensors

CLASS efemarai.assertions.NoVanishingTensors(FunctionAssertion)

Ensure that no tensor in the computational graph becomes almost all zeros.

`def __init__(self, threshold=1e-5)`

Initialize the assertion.

Parameters

• threshold (`float`) – if the mean of the absolute values of all tensor elements is less than this threshold then the tensor is considered ‘vanishing’.

Examples:

`ef.register_assertion(NoVanishingTensors(threshold=1e-3))`

## Activation Functions

Ensure that there are no dead ReLU layers in your model. In order to avoid storing previous ReLU outputs this assertion detects dead ReLU layers based on batch activations.

`def __init__(self, threshold=0.8)`

Initialize the assertion.

Parameters

• threshold (`float`) – if the fraction of inactive ReLU outputs per batch is greater than the threshold, then the layer is considered dead.

Examples:

`ef.register_assertion(NoDeadReLUs(threshold=0.5))`

### NoSaturatedSigmoidUnits

CLASS efemarai.assertions.NoSaturatedSigmoidUnits(FunctionAssertion)

Ensure that there are no saturated sigmoid units in your model.

`def __init__(self, threshold=6)`

Initialize the assertion.

Parameters

• threshold (`float`) – if the absolute value of the input to a sigmoid unit is greater than this threshold then the unit is considered saturated.

### NoSaturatedTanhUnits

CLASS efemarai.assertions.NoSaturatedTanhUnits(FunctionAssertion)

Ensure that there are no saturated tanh units in your model.

`def __init__(self, threshold=6)`

Initialize the assertion.

Parameters

• threshold (`float`) – if the absolute value of the input to a tanh unit is greater than this threshold then the unit is considered saturated.

## Loss Functions

### ValidInputsDiscreteNLLLoss

CLASS efemarai.assertions.ValidInputsDiscreteNLLLoss(TensorAssertion)

When using the NLL loss for discrete problems such as classification the input is probability mass and so its log cannot be positive.

### ValidInputsDiscretePoissonNLLLoss

CLASS efemarai.assertions.ValidInputsDiscretePoissonNLLLoss(TensorAssertion)

When using the Poisson NLL loss for discrete problems the input is probability mass and so its log cannot be positive. Takes into account the `log_input` input argument.

### ValidInputsDiscreteKLDivLoss

CLASS efemarai.assertions.ValidInputsDiscreteKLDivLoss(TensorAssertion)

When using the KL divergence loss for discrete problems the input is probability mass and so its log cannot be positive. Takes into account the `log_target` input argument.