Souvik Banerjee

ML Engineer

Biography

I recently finished my undergradute degree in International Insititute of Information technology, Hyderabad (IIIT-H) double majoring in computer science and Natural language processing(May 2023). During college, I was working at the MT-NLP lab with Prof. Manish Shrivastava as my main advisor and Bamdev Mishra (Microsoft), Pratik Jawanpuria (Microsoft) as my external advisors.

I am currently looking for project/research associate roles in those same fields that I have mentioned here. For the last one and a half year, I have been working as an ML Engineer and a Web designer for a startup company in Hyderbad which goes by the name of Mirelz Ai Private Limited.

Research Interests

Differential Geometry - Differentiable manifold Optimization theory, Lie Group theory and Information Geometry, Morse theory and Algebraic Geometry
Geometric Deep Learning - Topological Deep Learning
NLP - Textual Embeddings, Machine translation

latest posts

Apr 12, 2023	Spectahedron Manifold Word Embeddings

selected publications

ICON

Generalised Spherical Text Embedding

Souvik Banerjee, Bamdev Mishra, Pratik Jawanpuria, and 1 more author

In Proceedings of the 19th International Conference on Natural Language Processing (ICON), Dec 2022

Abs

This paper aims to provide an unsupervised modelling approach that allows for a more flexible representation of text embeddings. It jointly encodes the words and the paragraphs as individual matrices of arbitrary column dimension with unit Frobenius norm. The representation is also linguistically motivated with the introduction of a metric for the ambient space in which we train the embeddings that calculates the similarity between matrices of unequal number of columns. Thus, the proposed modelling and the novel similarity metric exploits the matrix structure of embeddings. We then go on to show that the same matrices can be reshaped into vectors of unit norm and transform our problem into an optimization problem in a spherical manifold for optimization simplicity. Given the total number of matrices we are dealing with, which is equal to the vocab size plus the total number of documents in the corpus, this makes the training of an otherwise expensive non-linear model extremely efficient. We also quantitatively verify the quality of our text embeddings by showing that they demonstrate improved results in document classification, document clustering and semantic textual similarity benchmark tests.
RepL4NLP

Word Embeddings as Tuples of Feature Probabilities

Siddharth Bhat, Alok Debnath, Souvik Banerjee, and 1 more author

In Proceedings of the 5th Workshop on Representation Learning for NLP, Jul 2020

Abs

In this paper, we provide an alternate perspective on word representations, by reinterpreting the dimensions of the vector space of a word embedding as a collection of features. In this reinterpretation, every component of the word vector is normalized against all the word vectors in the vocabulary. This idea now allows us to view each vector as an n-tuple (akin to a fuzzy set), where n is the dimensionality of the word representation and each element represents the probability of the word possessing a feature. Indeed, this representation enables the use fuzzy set theoretic operations, such as union, intersection and difference. Unlike previous attempts, we show that this representation of words provides a notion of similarity which is inherently asymmetric and hence closer to human similarity judgements. We compare the performance of this representation with various benchmarks, and explore some of the unique properties including function word detection, detection of polysemous words, and some insight into the interpretability provided by set theoretic operations.