Indexing in information retrieval

Hello. This blog is about indexing techniques used in information retrieval. I will try to cover some basic indexing techniques and also provide code for some of them. What is indexing? An index is a data structure that is created to efficiently store and retrieve information from a collection of documents or data. The index allows for quick lookup and retrieval of documents based on user queries or search terms. In search engines, web pages are indexed, when user enters a query, the engine finds the relevant documents quickly using the index. Below are some indexing techniques: ...

June 18, 2023 · 2 min

(GSoC) Week-2 Entity Linking - Intro

Hello! In this blog, we will discuss about entity linking, a process used to map recognized entities to resources in a knowledge base, like DBpedia. Background Entity linking is the process of linking or mapping entity mentions(the identified entities in a text) to actual resources in a knowledge base. This is an important step towards our ultimate goal of relation extraction. Entity mentions in text need not be exactly same as the entities in the knowledge base, so it is very essential that we map them to the correct entity(in the KB) so as to extract correct relations. It is an important step which acts as a bridge to connect unstructured text to knowledge bases. ...

June 13, 2023 · 2 min

(GSoC) Week-1 Named Entity Recognition

Hi! In this post, I will be talking about NER, a very popular and important task in NLP! Background Named Entity Recognition, which is a subtask of natural language processing (NLP), involves identifying and classifying named entities within a given text. The goal is to identify and categorize the entities present in text. This also forms a step in our goal of relation extraction. Entities present in text can be calssified(broadly) into a variaty of types like PERSON, ORGANIZATION, LOCATION, TIME, etc. It is easy for humans to identify and categorize entities from text. When we read or hear a sentence like "Messi is a football player of Argentina", we can easily identify that Messi is a person and Argentina is a location/country. Identifying these further helps us understand the relation between them, in this case, the relation is - player of. We can think of named entities as answers to wh questions like Who, What, When ,Where etc. in a sentence. But it is not as easy for a machine to understand and process text as we do. For this, we need either rule based methods or machine learning/deep learning approaches. ...

June 1, 2023 · 5 min

Accepted in GSoC 2023!

Hello and welcome! I am delighted to write my first blog post about my acceptance in GSoC 2023 with DBpedia. Throughout the summer, I will work on the Neural extraction framework project with DBpedia. Background: It all started last year when I came to know about GSoC. It was that time when I learned about open-source, and I liked the idea of anyone being able to contribute and change the codes of so many softwares we use every day. I found an organization and worked hard contributing and writing a proposal. Though I was not selected for GSoC, I added two new distance metrics to a vector database later that summer. Since then, I have kept looking for opportunities in open-source projects, especially those I use. ...

May 21, 2023 · 3 min