Deconstructing BERT, Part 2: Visualizing the Inner Workings of Attention

In this post, the author shows how BERT can mimic a Bag-of-Words model. The visualization tool from Part 1 is extended to probe deeper into the mind of BERT, to expose the neurons that give BERT its shape-shifting superpowers.

Go to the post on: KDnuggets RSS Feed.