10 posts tagged with "Python"
How Exactly Does Masking in Transformer Work
Masking is one of those concepts that is easy to wave your hands at but quite important if you want to implement the Transformer from…
BERT: A Detailed Guide to Clear Up All Your Confusions
Pre-requisites Understanding of Transformer, especially how masking works. (I strongly recommend Jay Alammar's article The Illustrated…
What does __getitems__ mean? Functions with Double Underscores in Python
While reading the PyTorch source code, I started to notice that many classes have functions named __xxx__ such as __init__ , __len__, or…
Byte-pair Encoding Algorithm
BPE (byte-pair encoding) is a good way (alternative to schemes such as one-hot encoding and Word2Vec pre-trained embeddings) to encode words…
Masking in the Transformer Explained
The Transformer is a landmark breakthrough in NLP that is explained quite well by Jay Alammar's article The Illustrated Transformer. This…
PyTorch scatter_ Function Explained
In PyTorch, is a function you can use to write the values in tensor into the tensor. The best way I found to think about this function is…
Re Tutorial: A Quick Python Start
Re, or Regex, stands for regular expression, which means "a sequence of characters that define a search pattern."1 It is particularly useful…
Binary Tree Level Order Traversal Python Solution
This problem can be better solved iteratively rather than recusively. The essential idea is that you can get all the next level nodes from…
Maximum Subarray - Kadane's Algorithm Explained
I was inspired to write this post after completing the Leetcode challenge and coming across the Kadane algorithm. It's a very interesting…
How to Run a Python Script in Your Node Backend
While javascript is awesome, you're not obligated to use it throughout your entire stack. There will come a time where it is more…