mxnet-seq2seq

Sequence to sequence learning in MXNet for an open-domain chatbot, using two LSTMs, a shared embedding layer, and bucketed decoding for variable sequence lengths.

The project implements sequence to sequence learning with MXNet for an open-domain chatbot, written in Python. The encoder-decoder architecture is credited to the Sequence to Sequence Learning with Neural Networks paper, and it uses two RNNs, both LSTMs, with one encoding the source sequence and the other decoding the target.

For NLP tasks the sequence is a natural language sentence, so the encoder and decoder share a word embedding layer rather than each learning its own. The implementation borrows from an LSTM bucketing approach, slightly modified, with the embedding layer reconstructed for this project. Bucketing is presented as a solution to arbitrary sequence lengths: zeros pad the encoding sequence to a fixed length, and buckets handle the decoding side, which keeps variable-length input practical instead of forcing every sample to the longest one. It is a pragmatic middle ground between padding everything and doing fully dynamic computation.

The README gives the tensor shapes directly. The embedding layer takes input of shape batch size by sequence length, and the LSTM encoder takes batch size by sequence length by embedding dimension. Those two shapes make the data flow easy to follow: sentences come in as token indices, become embedding vectors, and then feed the encoder LSTM. Beyond those numbers, the README does not go into training details, so the value here is mainly the architecture sketch.

The README keeps its scope to the architecture and the bucketing choices, with the lineage traced back to an LSTM bucketing approach and the original seq2seq paper.

• Source 1 · Project repository
• Source 2 · README