## Numerical Stability

### NoNansAssertion

CLASS efemarai.assertions.NoNansAssertion(TensorAssertion)

Ensure that no tensor contains any `NaN`

values.

Registered by default.

Examples:

ef.register_assertion(NoNansAssertion()) ef.deregister_assertion(NoNansAssertion)

### NoInfsAssertion

CLASS efemarai.assertions.NoInfsAssertion(TensorAssertion)

Ensure that no tensor contains any `Inf`

values.

Registered by default.

Examples:

ef.register_assertion(NoInfsAssertion()) ef.deregister_assertion(NoInfsAssertion)

### NoNonZeroGradientsAssertion

CLASS efemarai.assertions.NoNonZeroGradientsAssertion(TensorAssertion)

Ensure that all gradients contain only zeros before the backward pass is executed.

Registered by default.

Examples:

ef.register_assertion(NoNonZeroGradientsAssertion()) ef.deregister_assertion(NoNonZeroGradientsAssertion)

### NoExplodingTensors

CLASS efemarai.assertions.NoExplodingTensors(FunctionAssertion)

Ensure that no tensor in the computational graph has an element less / greater than a certain threshold.

`def __init__(self, threshold=1e5)`

Initialize the assertion.

Parameters

**threshold**(`float`

) – if a tensor has elements with absolute value greater than this threshold then the tensor is considered ‘exploding’.

Examples:

ef.register_assertion(NoExlodingTensors(threshold=1e2))

### NoVanishingTensors

CLASS efemarai.assertions.NoVanishingTensors(FunctionAssertion)

Ensure that no tensor in the computational graph becomes almost all zeros.

`def __init__(self, threshold=1e-5)`

Initialize the assertion.

Parameters

**threshold**(`float`

) – if the mean of the absolute values of all tensor elements is less than this threshold then the tensor is considered ‘vanishing’.

Examples:

ef.register_assertion(NoVanishingTensors(threshold=1e-3))

## Activation Functions

### NoDeadReLULayers

CLASS efemarai.assertions.NoDeadReLULayers(FunctionAssertion)

Ensure that there are no dead ReLU layers in your model. In order to avoid storing previous ReLU outputs this assertion detects dead ReLU layers based on batch activations.

`def __init__(self, threshold=0.8)`

Initialize the assertion.

Parameters

**threshold**(`float`

) – if the fraction of inactive ReLU outputs per batch is greater than the threshold, then the layer is considered dead.

Examples:

ef.register_assertion(NoDeadReLUs(threshold=0.5))

### NoSaturatedSigmoidUnits

CLASS efemarai.assertions.NoSaturatedSigmoidUnits(FunctionAssertion)

Ensure that there are no saturated sigmoid units in your model.

`def __init__(self, threshold=6)`

Initialize the assertion.

Parameters

**threshold**(`float`

) – if the absolute value of the input to a sigmoid unit is greater than this threshold then the unit is considered saturated.

### NoSaturatedTanhUnits

CLASS efemarai.assertions.NoSaturatedTanhUnits(FunctionAssertion)

Ensure that there are no saturated tanh units in your model.

`def __init__(self, threshold=6)`

Initialize the assertion.

Parameters

**threshold**(`float`

) – if the absolute value of the input to a tanh unit is greater than this threshold then the unit is considered saturated.

## Loss Functions

### ValidInputsDiscreteNLLLoss

CLASS efemarai.assertions.ValidInputsDiscreteNLLLoss(TensorAssertion)

When using the NLL loss for discrete problems such as classification the input is probability mass and so its log cannot be positive.

### ValidInputsDiscretePoissonNLLLoss

CLASS efemarai.assertions.ValidInputsDiscretePoissonNLLLoss(TensorAssertion)

When using the Poisson NLL loss for discrete problems the input is probability mass and so its log cannot be positive. Takes into account the `log_input`

input argument.

### ValidInputsDiscreteKLDivLoss

CLASS efemarai.assertions.ValidInputsDiscreteKLDivLoss(TensorAssertion)

When using the KL divergence loss for discrete problems the input is probability mass and so its log cannot be positive. Takes into account the `log_target`

input argument.