Home Applications AriticleSimilarity

AriticleSimilarity

This application is not supported by InterSystems Corporation. Please be notified that you use it at your own risk.
3.5
1 reviews
1
Awards
181
Views
0
IPM installs
3
4
Details
Releases
Reviews  (1)
Awards  (1)
Issues
Articles  (2)
This app has a demo View
Comparing article similarity through vector search

What's new in this version

Initial Release

AriticleSimilarity

Using text vectors for article similarity comparison,The generation of vectors used Python’s sense transformers package, and similarity calculation used Iris’s vector cosine calculation. The front-end uses bootstrap and CSP

Installation

1.docker

1.1 Pull the code to a local directory

 $ git clone https://github.com/ADAQIQI/AriticleSimilarity.git

1.2 Download the model and place it in the specified directory, which can be found in the https://huggingface.co/ Taking the BERT model as an example(If the CPU and memory configuration are not sufficient, other models such as MiniLM can be considered), download it and place it in the same directory

image

image

1.3 If using other models, it is necessary to modify the dockerfile file

image

Change to the name of the model folder you have created.

1.4 docker-compose up -d

If the model used is not BERT, it is necessary to Modify Article Similarity Vector’s GetEmbeddingPy Change the directory to the folder used in 1.2

image

2.IPM

 zpm install ariticlesimilarity

Reference 1.2 Place the model folder in the specified directory

install sentence-transformers

python3 -m pip install --target /usr/irissys/mgr/python sentence-transformers

How to use

Enter the homepage

http://localhost:52773/csp/user/Index.csp

Enter/upload the title and content of the article, with a demonstration text generated by Google Board

image

After clicking upload, wait for a period of time for the page to jump to paragraph similarity comparison(If the page reports an error due to a long return time, please wait for a while before refreshing the page and directly switch to the similarity comparison tab for similarity comparison). At the same time, the background will continue to split the article into sentences and compare them sentence by sentence

image

image

After refreshing the page, click the Sentence Similarity button, select the article title, and click the button to view the comparison of sentence similarity. Sentences with similarity higher than 0.7 (can be modified in the bo class) will be distinguished by color

image

Made with
Install
zpm install ariticlesimilarity
Version
1.0.009 May, 2024
Python package
sentence-transformers
Category
Technology Example
Works with
InterSystems IRISInterSystems Vector Search
First published
30 Apr, 2024