by Sam Shamsan

The man has always been fighting an ultimate inner war to better himself. This war within has created civilizations, cultures and regions. Since The Epic of Gilgamesh, the first book ever written that we know of, many books have been written. In this article, we will discuss…

Have you find yourself procrastinating on doing somthing you really want/have to do. Signing up to a gym and start working out, making new friends, starting your side project or even talking to that girl you find attractive.

What if i told that using this method of “minimum due” could…

Transfer learning aims to transfer learning from one domain to another. It’s motivated by the fact that building models from scratch comes with high cost in terms of time and resources.

Transfer learning uses the source domain as the knowledge base and transfers its knowledge to the newly targeted domain…

GPT is an autoregressive model that leverages a large linguistic information of unlabeled data and uses its representation to tackle many downstream tasks. The model has two stages of training, generative and discriminative. …

The need for a small and efficient model has always been there for many reasons such as resource and adaptation for experimental changes. The inference time is even more important since that is what the end user is experiencing. So the need for smaller and faster models is crucial.

A…

The first layer of the encoder architecture in transformers is the self attention. This layer take word embedding in word2vec or any other contextual embedding add to the positional embedding where it measures the words order by calculating the distance between the words.

These input sequences enter the self attention…

Decoding enhanced BERT with disentangled attention

The main motive behind it is that attention weight should be based not only on the content but the position as well. Instead of adding them together, the approach here is to separate them. …

Sam Shamsan

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store