Removed Demo
Local LLM suppor tin the form of phi3:mini has been added to show the possibility of a fully local environment
Introduction
InterSystems IRIS Integration
IRIS RAG Interface Overview
Embedding Options
Attribution
Quickstart
Backend
Frontend
License
Retrieval-Augmented Generation (RAG) is a powerful AI model that combines the strengths of retrieval-based and generative models. It leverages a pre-trained language model to generate responses based on retrieved documents, enabling more accurate and context-aware answers.
RAG enhances the quality of AI-generated responses by incorporating relevant information from existing documents. This approach improves the accuracy and relevance of the generated content, making it ideal for a wide range of applications, from chatbots to content creation.
This application leverages InterSystems IRIS as its backend database to efficiently store and manage documents. IRIS is a powerful, high-performance data platform known for its robust data management capabilities.
The application utilizes the IRIS Django template for seamless integration with Django, allowing for effective interaction with the IRIS database. Additionally, it incorporates the Llama-IRIS template, which is built on SQLAlchemy, to handle advanced document management and querying.
The application employs llama_index to manage the Retrieval-Augmented Generation (RAG) workflow, including handling queries, retrieving relevant documents, and generating responses based on the stored data.
Documents are stored in the IRIS database using the native iris $VECTOR data type. This specialized data type supports efficient vector storage and retrieval, which is crucial for handling high-dimensional data used in machine learning and document processing.
The frontend of the application is built with Vue.js and styled using Tailwind CSS, providing a modern and responsive user interface for interacting with the application.
The application interface provides several key functionalities:
Here’s how you can navigate and utilize the key components of the interface:
Choose the AI model for processing your queries. This dropdown allows you to select from available models to tailor the query results to your needs.
Select the embedding type (e.g., OpenAI or BGA-Large) to influence how documents are represented in vector space.
Set the temperature parameter (range 0-2) to control the randomness of the AI’s responses. Lower values make the output more deterministic.
Define the number of top similar documents to consider in the query results. Adjust this value to determine how many results are sent to the LLM to answer the query.
Set the threshold for document similarity to filter out less relevant results. This helps in retrieving documents closely matching the query.
Enter the text of your query to search through documents. If left blank, a default query will be used: “Describe this document”.
Provide a name for the new document you wish to add. If omitted, the document text will be ignored.
Input the content of the new document here. This field is crucial for adding new documents to the system.
Select from a list of existing documents for querying or managing. This multi-select input lets you query multiple documents at once.
Use this button to execute queries or add new documents. The results or updates will be displayed accordingly.
Reset all input fields and selections to their default states, clearing the form for a new entry.
Remove selected documents from the backend, including their vector embeddings. This action is irreversible, so proceed with caution.
View the results of your queries, including document content and citations, displayed here.
Displays a list of all currently available documents for quick reference and selection.
When configuring the document management and querying features, you have the option to choose between local embeddings and API-based embeddings. Here’s a detailed comparison of the available embedding options:
bge-large is an embedding model provided by BAAI on Hugging Face. It offers a free and open-source solution for generating document embeddings locally.
OpenAI provides embeddings via its API, offering a powerful and scalable solution for generating high-quality document embeddings.
Local embeddings offer a cost-effective and flexible solution for offline processing, while API-based embeddings provide advanced capabilities and scalability for cloud-based applications.
The vectorstore.py file in this project that implements the IrisVectorStore object with sqlalchemy was written mostly by “Dmitry Maslennikov dmitry@caretdev.com” in the llama-iris package here: https://github.com/caretdev/llama-iris/ which is why llama-iris is included in the requirements.txt but not used in the package. It’s dependencies are required and better defined in the llama-iris package which I hope to integrate into this project in the future after small updates are applied. For the sake of speed in creating this project for the InterSystems 2024 Python Contest, I brought the code in directly to have full control over the vector storage functionality.
Clone the Repository:
git clone https://github.com/mindfulcoder49/iris-django-template.git
cd iris-django-template
Create .env file:
OPENAI_API_KEY=sk-dkjhf...
IRIS_CONNECTION_STRING=iris://SuperUser:SYS@localhost:1972/IRISAPP
Run docker-compose build
Change the permissions on the repository folder and its contents
chmod -R 777 .
or
chmod -R 777 ../iris_django_template
Run docker-compose up
Access the Application:
http://localhost:53795/django/documents/
http://localhost:53795/csp/irisapp/EnsPortal.ProductionConfig.zen
login: SuperUser/SYSapp/app/urls.py
from django.contrib import admin from django.urls import path, include from rest_framework import routers from interop.views import index as interop_index from documents.views import index as documents_index, get_documents, delete_documents from django.views.generic import TemplateView
router = routers.DefaultRouter()
urlpatterns = [
path('admin/', admin.site.urls),
path('interop/', interop_index),
path('api/documents/', documents_index),
path('api/document_names/', get_documents),
path('api/documentsdelete/', delete_documents),
path('documents/', TemplateView.as_view(template_name='index.html')),
]
app/documents/vectorstore.py
app/documents/models.py
app/documents/views.py
This Django module provides several functionalities for managing and querying documents using a vector store and an AI model. It offers endpoints for indexing new documents, querying existing documents, retrieving a list of documents, and deleting documents from the database and vector store. The code integrates with OpenAI and utilizes LlamaIndex for handling vector storage and querying.
index(request)
Description: Handles document indexing and querying. It:
Request Payload:
query_text
: The query to be executed on existing documents.document_text
: The text content of the new document.document_name
: The name of the new document.model_name
: The model to be used for querying (default: gpt-4o-mini
).embed_type
: The type of embedding to use.temperature
: The temperature setting for the model (default: 0.5
).top_k_similarity
: The number of top similar documents to consider.similarity_threshold
: The threshold for document similarity.selected_documents
: List of document names to query.Response:
responses
: A dictionary with document names as keys and their responses and citations as values.existing_documents
: List of all documents stored in the database.get_documents(request)
Description: Retrieves and returns the names of all documents currently stored in the database.
Response:
existing_document_names
: List of names of all documents stored in the database.delete_documents(request)
Description: Deletes specified documents from both the database and the vector store.
Request Payload:
document_names
: List of document names to be deleted.Response:
status
: Success message if documents are deleted successfully.error
: Error message if the operation fails.Django
: Web framework.logging
: For logging errors and debugging information.dotenv
: For loading environment variables.json
: For parsing JSON data.openai
: For integrating with OpenAI’s language models.llama_index
: For vector storage and querying.os
: For accessing environment variables and file operations.app/documents/existing_document_handling.py
get_clean_name(name)
: Sanitizes and formats a document name for use in the vector store.query_existing_document(name, query)
: Queries a document in the vector store and returns the response.get_clean_name(name)
Description: Cleans and formats a document name by converting it to lowercase and replacing whitespace with underscores.
Parameters:
name
(str): The original document name.Returns:
Example:
clean_name = get_clean_name("My Document Name")
# Output: "my_document_name"
query_existing_document(name, query)
Description: Queries a document in the vector store and returns the response. Connects to the vector store, loads the document’s index, and processes the query.
Parameters:
name
(str): The name of the document to query.query
(str): The query text to be executed against the document.Returns:
response
(ResponseType): The response object containing the result of the query.Raises:
ValueError
: If the connection string is not set or if loading the index fails.Example:
response = query_existing_document("my_document_name", "What is the content?")
print(response.response)
# Output: (response from the vector store)
gpt-4o-mini
).0.5
).responses
and existing_document_names
.This project is licensed under the MIT License. See the https://github.com/mindfulcoder49/iris-django-template/blob/main/LICENSE file for details.