Home/ Questions/What is the difference between tf.nn.sparse_softmax_cross_entropy_with_logits vs tf.nn.softmax_cross_entropy_with_logits?
Next
Answered
Marley White
  • 33
Asked: 2022-05-18T15:36:36+00:00 In:

What is the difference between tf.nn.sparse_softmax_cross_entropy_with_logits vs tf.nn.softmax_cross_entropy_with_logits?

  • 33

. Advertisement .

..3..

. Advertisement .

..4..

The difference between tf.nn.sparse_softmax_cross_entropy_with_logits vs tf.nn.softmax_cross_entropy_with_logits? I am learning about it. Can someone share with me your knowledge and experience?

softmax_cross_entropy_with_logits
Share

2 Answers

  1. Best Answer
    Expert

    It’s convenient to have two separate functions that yield the same result.

    The distinction is straightforward:

    • Labels must have the shape [batch_size] and the dtype int32 or int64 for sparse_softmax_cross_entropy_with_logits. Each label is an int in range [0, num_classes-1].
    • Labels must be of the shape [batch_size, num_classes] and dtype float32 or float64 for softmax_cross_entropy_with_logits.

    softmax_cross_entropy_with_logits labels are the hottest form of sparse_softmax_cross_entropy_with_logits labels.

    Another minor distinction is that you can use -1 as a label with sparse_softmax_cross_entropy_with_logits to have loss 0 on this label.

    • 22

  2. To add to the accepted answer, I’d like to add two things that you can also find within TF documentation.

    First:

    tf.nn.softmax_cross_entropy_with_logits

    NOTE: While the classes are mutually exclusive, their probabilities
    need not be. All that is required is that each row of labels is a
    valid probability distribution. If they are not, the computation of
    the gradient will be incorrect.

    Second:

    tf.nn.sparse_softmax_cross_entropy_with_logits

    NOTE: For this operation, the probability of a given label is
    considered exclusive. That is, soft classes are not allowed, and the
    labels vector must provide a single specific index for the true class
    for each row of logits (each minibatch entry).

    • 12

Leave an answer

You must login to add an answer.

Need An Account,