Loading…
Loading…
This project implements an encoder-decoder Transformer in PyTorch and adds training, tokenization and translation interfaces. It is a study of the mechanism described in Attention Is All You Need: positional encoding, multi-head attention, feed-forward layers and autoregressive decoding.
The current configuration lists eleven Indic languages: Assamese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil and Telugu. Each configured pair translates from that Indic language into English. This does not establish an English-to-Indic or every-language-to-every-language translator.
The configuration selects Samanantar and dataset_fraction=0.01. It specifies five epochs, batch size 32, model width 512 and sequence length 350, with warmup and label smoothing. These are source settings. They are not evidence that all configured runs completed or that the full corpus was used.
The source tokenizer maps text to token IDs. The encoder computes contextual representations for the source sequence. The decoder uses masked self-attention and cross-attention to predict the next target token. Training compares those predictions with shifted target tokens.
At inference, decoding repeatedly appends a predicted token until the stop condition or length limit is reached. This is a working implementation pattern to inspect and test; fluent translations still depend on a trained checkpoint and the data used to train it.
The repository includes model, dataset and training code, a Streamlit interface and an API server. The API searches for a saved weights checkpoint at startup. If loading fails, translation requests return an unavailable-model response. A source checkout without weights is not a deployed translation service.
Pretrained weights and a scored translation report are not bundled in the inspected repository tree. The interface does not validate multilingual quality or scalability merely by accepting input. A completed evaluation needs the checkpoint, tokenizer versions, held-out corpus, decoding settings and per-language scores, with examples of failures as well as successes.
I implemented the Transformer components and the surrounding training and inference workflow. PyTorch supplies tensor operations and automatic differentiation; Samanantar supplies the parallel data; the original paper supplies the architecture.
The next reproducible comparison is a fixed subset and split with a saved baseline checkpoint, logged training settings and held-out translation scores per language. It should test checkpoint reload and unavailable-model behavior before describing the service as ready.
Read the detailed Transformer walkthrough
The corrected decoder allows each position to attend to itself and earlier positions, and blocks future positions. It also keeps the source batch dimension during decoding. The interface accepts the selected Indic source language and produces English, which matches the trained direction.
Four regression tests pass. They check future-token independence, the allowed attention mask, batched source decoding and the interface language direction. The same suite fails against the original pinned source. The transformer fixture runs the actual model code; the widget layer is mocked. These checks do not establish trained translation quality.
Download the uncommitted source correction and tests. Read the source and file-hash manifest. The patch applies to the pinned revision.