Building a Movie Recommendation App with ScyllaDB Vector Search

Use ScyllaDB to perform semantic search across movie plot descriptions We built a sample movie recommendation app to showcase ScyllaDB’s new vector search capabilities. The sample app gives you a simple way to experience building low-latency semantic search and vector-based applications with ScyllaDB. Join the Vector Search Early Access Program In this post, we’ll show how to perform semantic search across movie plot descriptions to find movies by meaning, not keywords. This example also shows how you can add ScyllaDB Vector Search to your existing applications.    Before diving into the application, let’s clarify what we mean by semantic search and provide some context about similarity functions. About vector similarity functions Similarity between two vectors can be calculated in several ways. The most common methods are cosine similarity, dot product (inner product), and L2 (Euclidean) distance. ScyllaDB Vector Search supports all of these functions. For text embeddings, cosine similarity is the most often used similarity function. That’s because, when working with text, we mostly focus on the direction of the vector, rather than its magnitude. Cosine similarity considers only the angle between the vectors (i.e., the difference in directions) and ignores the magnitude (length of the vector). For example, a short document (1 page) and a longer document (10 pages) on the same topic will still point in similar directions in the vector space even though they are different lengths. This is what makes cosine similarity ideal for capturing topical similarity. Cosine similarity formula In practice, many embedding models (e.g., OpenAI models) produce normalized vectors. Normalized vectors all have the same length (magnitude of 1). For normalized vectors, cosine similarity and the dot product return the same result. This is because cosine similarity divides the dot product by the magnitudes of the vectors, which are all 1 when vectors are normalized. The L2 function produces different distance values compared to the dot product or cosine similarity, but the ordering of the embeddings remains the same (assuming normalized vectors). Now that you have a better understanding of semantic similarity functions, let’s explain how the recommendation app works. App overview The application allows users to input what kind of movie they want to watch. For example, if you type “American football,” the app compares your input to the plots of movies stored in the database. The first result is the best match, followed by other similar recommendations. This comparison uses ScyllaDB Vector Search. You can find the source code on GitHub, along with setup instructions and a step-by-step tutorial in the documentation. For the dataset, we are reusing a TMDB dataset available on Kaggle. Project requirements To run the application, you need a ScyllaDB Cloud account and a vector search enabled cluster. Right now, you need to use the API to create a vector search enabled cluster. Follow the instructions here to get started! The application depends on a few Python packages: ScyllaDB Python driver – for connecting and querying ScyllaDB. Sentence Transformers – to generate embeddings locally without requiring OpenAI or other paid APIs. Streamlit – for the UI. Pydantic – to make working with query results easier. By default, the app uses the all-MiniLM-L6-v2 model so anyone can run it locally without heavy compute requirements. Other than ScyllaDB Cloud, no commercial or paid services are needed to run the example. Configuration and database connection A config.py file stores ScyllaDB Cloud credentials, including the host address and connection details. A separate ScyllaDB helper module handles the following: Creating the connection and session Inserting and querying data Providing helper functions for clean database interactions Database schema The schema is defined in a schema.cql file, executed when running the project’s migration script. It includes: Keyspace creation (with a replication factor of 3) Table definition for movies, storing fields like release_date, title, genre, and plot Vector search index Schema highlights: `plot` – text, stores the movie description used for similarity comparison. `plot_embedding` – vector, embedding representation of the plot, defined using the vector data type with 384 dimensions (matching the Sentence Transformers model). `Primary key` – id as the partition key for efficient lookups querying by id CDC enabled – required for ScyllaDB vector search. `Vector index` – an Approximate Nearest Neighbor (ANN) index created on the plot_embedding column to enable efficient vector queries. The goal of this schema is to allow efficient search on the plot embeddings and store additional information alongside the vectors. Embeddings An Embedding Creator class handles text embedding generation with Sentence Transformers. The function accepts any text input and returns a list of float values that you can insert into ScyllaDB’s `vector` column. Recommendations implemented with vector search The app’s main function is to provide movie recommendations. These recommendations are implemented using vector search. So we create a module called recommender that handles Taking the input text Turning the text into embeddings Running vector search Let’s break down the vector search query: SELECT * FROM recommend.movies ORDER BY plot_embedding ANN OF [0.1, 0.2, 0.3, …] LIMIT 5; User input is first converted to an embedding, ensuring that we’re comparing embedding to embedding. The rows in the table are ordered by similarity using the ANN operator (ANN OF). Results are limited to five similar movies. The SELECT statement retrieves all columns from the table. In similarity search, we calculate the distance between two vectors. The closer the vectors in vector space, the more similar their underlying content. Or, in other words, a smaller distance suggests higher similarity. Therefore, an ORDER BY sort results in ascending order, with smaller distances appearing first. Streamlit UI The UI, defined in app.py, ties everything together. It takes the user’s query, converts it to an embedding, and executes a vector search. The UI displays the best match and a list of other similar movie recommendations. Try it yourself! If you want to get started building with ScyllaDB Vector Search, you have several options: Explore the source code on GitHub Use the README to set up the app on your computer Follow the tutorial to build the app from scratch And if you have questions, use the forum and we’ll be happy to help.