Configuring RAG pipelines requires iteration across different parameters ranging from pre-processing loaders and chunkers, to the actual embedding model being used. To assist in testing different configurations, Neum AI provides several tools to test, evaluate and compare pipelines.

Today, keyword-based and full-text search are not enough. The software industry is moving towards a new kind of search that goes beyond keywords and fuzzy-matching, it's moving towards semantic search. Delve into how you can take advantage of this new shift using Neum AI

Real-time synchronization of embeddings into vector databases is now trivial! Learn how to create a real-time Retrieval Augmented Generation pipeline with Neum and Supabase.

Following the release of Neum AI framework, an open-source project to build large scale RAG pipelines, we explore how to get started building with the framework in a multi-part series. In this blog, we go deep into leveraging distributed architecture tools like Celery and Redis Queues to build a solution to handle large datasets.

Following the release of Neum AI framework, an open-source project to build large scale RAG pipelines, we explore how to get started building with the framework in a multi-part series.

Neum AI provides tools to help you process structured data ahead of generating embeddings and loading it into vector databases. In this blog, we showcase the semantic selectors that help you choose what data (if any) from your structured data is worth embedding.

In this blog post we will go into some technical and architectural details of how we do this at Neum AI, specifically on how we did this for a pipeline syncing 1 billion vectors.

Q&A with a document is probably the most common scenario most developers think about today when it comes to LLMs. In this blog we explore what it takes to build that scenario at scale. Because the only thing cooler than doing Q&A with one document, it is to Q&A with thousands of them.

Neum AI enables AI engineers to connect their data sources to their LLMs through Retrieval Augmented Generation (RAG). Neum AI supports a variety of data sources that you can pull from as well as vector databases where you can have vectors stores to then do retrieval. Today, we are announcing support for SingleStore as both a data source and vector database. SingleStore allows you to keep all your data in a single place while leveraging the power of vector embeddings and RAG. Neum AI makes it easy to generate vector embeddings for the data and connect everything together.

Pre-processing documents before embedding them continue to be a challenge and an important step in ensuring the quality of RAG. At Neum, we wanted to create a simple playground to help developers test out their pre-processing steps with their documents. The playground is forked off the Langchain text splitter explorer. We will be further adding features to the playground to help add more loaders, metadata extractors and semantic splitters for different types of documents.

Neum AI introduces a context-aware text splitting feature that improves the efficiency of Large Language Models in handling specialized content like SEC filings or templated contracts. The new feature enhances Retrieval Augmented Generation (RAG) by allowing custom strategies for text segmentation, boosting retrieval quality and application performance.

A couple days ago, we released ElectionGPT.ai with the goal of showcasing experiences that use LLMs with Retrieval Augmented Generation (RAG) and are grounded on factual data from a variety of sources. In this blog, we would like to explore how we built it using Neum AI and the learnings we had in the process.

Data is the most important asset when building AI applications. Having up-to-date context in prompts or when doing semantic search is crucial. With Neum, we not only synchronize source data with vector stores, but we do so efficiently, saving costs by only vectorizing the changed data.

Spreadsheets and tabular data sources are commonly used and hold information that might be relevant for LLM based applications. In this blog we explore the different types of approaches towards connecting this data to your application. We deep dive into generating vector embeddings from this data taking into consideration the different types of date that a single spreadsheet or tabular data source might hold.