LSTM usually can handle about 200 length sequence effectively. However, if you should handle more than 200, for example 2000 length, how to do?
How Long Sequence Can be Processed Effectively by LSTM? – LSTM Tutorial
In this tutorial, we will introduce IndRNN, which is the tensorflow implementation of the paper: Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN.
How is IndRNN better than LSTM?
The IndRNN model modifies the computation of hidden state and output, it is:
The performance of IndRNN
IndRNN can handle longer sequence than LSTM, here is a comparative result.
From the result, we can find: LSTM perform worse when the sequence length is more than 200. However, IndRNN has a good performance. Even if the sequence length is 5000, IndRNN is a suitable performance.
How to use IndRNN
Step 1: Copy ind_rnn_cell.py into your project
Step 2: Use code below
from ind_rnn_cell import IndRNNCell # Regulate each neuron's recurrent weight as recommended in the paper recurrent_max = pow(2, 1 / TIME_STEPS) cell = MultiRNNCell([IndRNNCell(128, recurrent_max_abs=recurrent_max), IndRNNCell(128, recurrent_max_abs=recurrent_max)]) output, state = tf.nn.dynamic_rnn(cell, input_data, dtype=tf.float32) ...