Essays and notes on recommendation systems, search, relevance, and building AI products by Tullie Murrell.https://tullie.ai/en-ushttps://tullie.ai/blog/spatial-discovery-building-a-latent-space-map/https://tullie.ai/blog/spatial-discovery-building-a-latent-space-map/Most blog discovery is chronological, popular, or search-driven. I built a 2D latent space map of 85 posts to see whether spatial navigation could offer a better bird's-eye view of the corpus.Mon, 15 Jun 2026 00:00:00 GMTEnghttps://tullie.ai/blog/agent-trajectories-answer-rates/https://tullie.ai/blog/agent-trajectories-answer-rates/A new paper introduces LRAT, a framework that fine-tunes dense retrievers on agent trajectories instead of human clicks. We summarize how it works, why agent signals differ from human search, and what the reported gains mean for production agents.Mon, 04 May 2026 00:00:00 GMTResearchhttps://tullie.ai/blog/hnsw-explained-the-algorithm-powering-fast-vector-search/https://tullie.ai/blog/hnsw-explained-the-algorithm-powering-fast-vector-search/If you've set up a vector database recently, you've seen it. Somewhere in the configuration — maybe when creating an index in Pinecone, or scanning Weaviate's schema options, or reading Qdrant's collection settings — there were the four letters HNSW. You probably selected it because it was the default or the recommended option, made a mental note to understand it properly later, and moved on. This is that article.Thu, 30 Apr 2026 00:00:00 GMTEngIRhttps://tullie.ai/blog/grep-vs-vector-db-retrieval/https://tullie.ai/blog/grep-vs-vector-db-retrieval/Keyword retrieval keeps winning in production for reasons that have little to do with benchmark leaderboard scores. This post explains when grep beats vectors, and why.Thu, 23 Apr 2026 00:00:00 GMTEngIRhttps://tullie.ai/blog/netflix-homepage-ranking-playbook/https://tullie.ai/blog/netflix-homepage-ranking-playbook/Learn how to build a fully personalized homepage like Netflix — where the rows themselves are personalized, not just the content inside them — using attribute ranking to surface the best genres per user, AI Views to generate micro-genres, and ShapedQL score ensembles to fill...Thu, 19 Mar 2026 00:00:00 GMTIRhttps://tullie.ai/blog/pinterest-related-pins-playbook/https://tullie.ai/blog/pinterest-related-pins-playbook/Learn how to build image-based recommendation systems like Pinterest's Related Pins using CLIP embeddings for visual similarity, AI Views for automatic image understanding, and ShapedQL score ensembles to blend visual, collaborative, and semantic signals — with full code...Thu, 12 Mar 2026 00:00:00 GMTIRhttps://tullie.ai/blog/spotify-discover-weekly-playbook/https://tullie.ai/blog/spotify-discover-weekly-playbook/Learn how to build personalized music recommendations like Spotify's Discover Weekly by blending collaborative filtering (ELSA) with content enrichment (genre/mood extraction), combining multiple signals in ShapedQL for hybrid recommendations.Mon, 09 Mar 2026 00:00:00 GMTIRhttps://tullie.ai/blog/ranking-architecture-part-5-feedback/https://tullie.ai/blog/ranking-architecture-part-5-feedback/Welcome to the final post in our series on the anatomy of modern recommender systems. Over the last four parts, we've deconstructed the online request path, following a user's request from the initial billions of items all the way to a final, ranked page.Mon, 02 Mar 2026 00:00:00 GMTIRhttps://tullie.ai/blog/beyond-the-hashing-trick-the-math-of-scaling-to-100m-ids-in-production/https://tullie.ai/blog/beyond-the-hashing-trick-the-math-of-scaling-to-100m-ids-in-production/If you follow machine learning today, you've been told that tokenization is a solved problem. In the world of Natural Language Processing (NLP), we have Byte Pair Encoding (BPE) or WordPiece. These algorithms compress the infinite complexity of human language into a neat, static vocabulary of roughly 50,000 to 100,000 tokens. This "dictionary" fits comfortably in a few megabytes of system RAM, and the resulting embedding weights take up a tiny fraction of a modern GPU's VRAM.Thu, 05 Feb 2026 00:00:00 GMTResearchIRhttps://tullie.ai/blog/building-the-relevance-layer/https://tullie.ai/blog/building-the-relevance-layer/Why retrieval and relevance are becoming the most important infrastructure in modern AI products, and what we're building at Shaped to solve it.Thu, 15 Jan 2026 00:00:00 GMTShapedIRhttps://tullie.ai/blog/why-i-built-a-database-for-relevance/https://tullie.ai/blog/why-i-built-a-database-for-relevance/After five years at Meta and three years building Shaped, I think relevance infrastructure should work like a database: declarative, composable, and fast enough for humans and agents.Thu, 11 Dec 2025 00:00:00 GMTShapedIRhttps://tullie.ai/blog/modeling-behavior-as-language-the-next-era-of-recommendations/https://tullie.ai/blog/modeling-behavior-as-language-the-next-era-of-recommendations/A major shift is underway in recommender systems, moving from traditional Two-Tower and DLRM models to a new paradigm that treats user behavior as a language. This approach models a user's sequence of interactions, such as clicks and purchases, allowing Transformer-based models to predict the next action with a more nuanced understanding of intent. While this evolution offers powerful capabilities for capturing dynamic user preferences, it also introduces significant new engineering challenges in managing inference costs, adapting feature stores for sequential data, and solving for new user cold-starts.Thu, 13 Nov 2025 00:00:00 GMTIRhttps://tullie.ai/blog/scaling-laws-beyond-llms-the-future-of-search-and-recommendations/https://tullie.ai/blog/scaling-laws-beyond-llms-the-future-of-search-and-recommendations/When people talk about scaling laws in AI, they usually mean one thing: language models. The empirical laws first quantified in Kaplan et al. (2020) showed that loss scales predictably as a power law with model size, dataset size, and compute budget. Train a bigger transformer on more text, and performance improves, up to the limits of optimization and overfitting.Thu, 13 Nov 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/ranking-infrastructure-part-1-serving/https://tullie.ai/blog/ranking-infrastructure-part-1-serving/Welcome to a new, hands-on series for builders. In our previous series, "Anatomy of a Modern Ranking Architectures," we deconstructed the conceptual blueprint of the multi-stage ranking architecture. We followed the logic of a request from retrieval to scoring to the final ordered page. Now, we shift from the "what" to the "how."Mon, 03 Nov 2025 00:00:00 GMTEnghttps://tullie.ai/blog/ranking-infrastructure-part-2-data-layer/https://tullie.ai/blog/ranking-infrastructure-part-2-data-layer/Welcome back to our series on the infrastructure of modern ranking systems. In Part 1, we designed the online serving layer: a set of decoupled, scalable microservices orchestrated by Kubernetes to handle real-time requests. We built the engine of our ranking system.Mon, 03 Nov 2025 00:00:00 GMTEngIRhttps://tullie.ai/blog/ranking-infrastructure-part-3-mlops/https://tullie.ai/blog/ranking-infrastructure-part-3-mlops/Welcome to the final post in our series on ranking infrastructure. We have the serving layer and the data layer in place; now we need the MLOps backbone that connects raw data to production models.Mon, 03 Nov 2025 00:00:00 GMTEnghttps://tullie.ai/blog/ranking-architecture-part-1/https://tullie.ai/blog/ranking-architecture-part-1/If you look under the hood of recommendation systems at Netflix, YouTube, or Amazon, you won't find identical models, but you will find a remarkably similar architectural blueprint. This multi-stage ranking system is the industry's shared solution to a fundamental engineering problem: how to find the few best needles in an ever-growing haystack, and do it in milliseconds. This is the first in a series of posts where we will deconstruct this blueprint. We'll go beyond high-level funnel diagrams and dive into the practical components, engineering trade-offs, and model architectures required to build a modern recommender system.Mon, 13 Oct 2025 00:00:00 GMTIRhttps://tullie.ai/blog/ranking-architecture-part-2-retrieval/https://tullie.ai/blog/ranking-architecture-part-2-retrieval/Welcome back to our series on the anatomy of modern recommender systems. In our first post, we established the multi-stage architecture as the industry-standard blueprint for balancing relevance, latency, and cost. We framed it as a system of cascading approximations, designed to efficiently identify the best items from a massive catalog. Today, we're diving deep into the first and arguably most critical part of this blueprint: The Retrieval Stage.Mon, 13 Oct 2025 00:00:00 GMTIRhttps://tullie.ai/blog/ranking-architecture-part-3-scoring/https://tullie.ai/blog/ranking-architecture-part-3-scoring/Welcome back to our series on the anatomy of modern recommender systems. In Part 1, we introduced the multi-stage architecture as a blueprint for balancing relevance, latency, and cost. In Part 2, we explored the Retrieval Stage, where we used an ensemble of strategies to generate a high-recall candidate set of about a thousand items.Mon, 13 Oct 2025 00:00:00 GMTIRhttps://tullie.ai/blog/ranking-architecture-part-4-ordering/https://tullie.ai/blog/ranking-architecture-part-4-ordering/Welcome back to our series on the anatomy of modern recommender systems. So far, we've deconstructed the core machine learning pipeline.Mon, 13 Oct 2025 00:00:00 GMTIRhttps://tullie.ai/blog/youtube-semantic-ids-ctr-lift/https://tullie.ai/blog/youtube-semantic-ids-ctr-lift/TL;DR: The shift from massive embedding tables to generative retrieval with Semantic IDs is accelerating. YouTube's new PLUM framework represents the next evolution, using an adapted LLM and enhanced 'SID-v2' to achieve a +4.96% Panel CTR lift for Shorts in live A/B tests. This deep dive explains how they did it.Fri, 10 Oct 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/building-a-hackernews-for-you-feed/https://tullie.ai/blog/building-a-hackernews-for-you-feed/TL;DR: The HackerNews top feed felt stale, so I built a personalized For You feed in a weekend using Lovable and Shaped. See it at hn.shaped.ai.Tue, 23 Sep 2025 00:00:00 GMTShapedIRhttps://tullie.ai/blog/beyond-keywords-how-to-rank-for-vibe-in-your-marketplace/https://tullie.ai/blog/beyond-keywords-how-to-rank-for-vibe-in-your-marketplace/Why is it so hard to find things you love on most marketplaces? Because users don't think in keywords, but platforms do. " They're thinking about a feeling.Thu, 18 Sep 2025 00:00:00 GMTIRhttps://tullie.ai/blog/vector-bottleneck-embedding-retrieval/https://tullie.ai/blog/vector-bottleneck-embedding-retrieval/DeepMind’s latest paper formalizes a long-suspected limitation of embedding-based retrieval: single-vector models cannot scale to combinatorial query complexity, no matter how large the dimension. The result reframes hybrid and multi-vector approaches, not as patches, but as necessary architectures for retrieval at scale.Mon, 08 Sep 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/dual-flow-generative-ranking/https://tullie.ai/blog/dual-flow-generative-ranking/TL;DR: Meta's generative recommender (MetaGR) is powerful but slow. Researchers from Meituan and top universities just dropped DFGR, a dual-stream architecture that's 2x faster at training and 4x faster at inference, while also beating MetaGR and heavily-engineered industrial models on ranking accuracy.Thu, 28 Aug 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/research-to-production-recommendations/https://tullie.ai/blog/research-to-production-recommendations/If you’ve ever tried to take a promising machine learning experiment from an offline notebook to a live A/B test, you know the pain. Weeks, sometimes months, pass between proving an idea works and actually seeing it in front of users. Internal handoffs, infrastructure gaps, and competing priorities all slow you down. And by the time your experiment goes live, the opportunity may have shifted, or worse, been shelved altogether. At Shaped, we’ve been thinking deeply about this “research-to-production gap” and how to eliminate it for recommendation systems. Here’s what we’ve learned.Wed, 13 Aug 2025 00:00:00 GMTResearchShapedEnghttps://tullie.ai/blog/measuring-personalization-are-your-recommendations-truly-unique/https://tullie.ai/blog/measuring-personalization-are-your-recommendations-truly-unique/Accuracy metrics like NDCG or Precision@K tell you how relevant your recommendations are, but not how unique they are to each user. This post explores the Personalization Score (inter-list diversity), which measures how different users' recommendation lists are. A high score...Wed, 06 Aug 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/renttherunway-dataset-deep-dive-into-fashion-fit-context-and-recommendation-challenges/https://tullie.ai/blog/renttherunway-dataset-deep-dive-into-fashion-fit-context-and-recommendation-challenges/Online fashion retail faces unique challenges, moving beyond simple preference prediction. Accurately recommending clothing requires understanding complex factors like fit, body type, and the context of use. The RentTheRunway (RTR) dataset emerges as a crucial and fascinating resource in this domain, offering rich data for researchers and data scientists tackling these fashion recommendation problems. This article provides a comprehensive overview of the RentTheRunway dataset, its unique characteristics, importance, and applications in building better recommendation systems.Tue, 05 Aug 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/where-matters-location-feature-engineering-for-search-recs/https://tullie.ai/blog/where-matters-location-feature-engineering-for-search-recs/Location is more than just coordinates, it’s a powerful signal for making search and recommendation systems more relevant. This post explores how proximity, regional preferences, delivery constraints, and geo-targeting can all be encoded into machine learning models through smart feature engineering. From geohashing and distance calculations to region embeddings and hierarchical modeling, we break down the core techniques and show how platforms like Shaped streamline the entire process, turning raw location data into real-time, personalized ranking power.Mon, 04 Aug 2025 00:00:00 GMTIRhttps://tullie.ai/blog/lambdamart-explained-the-workhorse-of-learning-to-rank/https://tullie.ai/blog/lambdamart-explained-the-workhorse-of-learning-to-rank/LambdaMART is one of the most widely used algorithms in Learning-to-Rank, powering the ranking logic behind search engines, recommendation systems, and e-commerce platforms. By combining gradient boosting trees (MART) with metric-aware optimization from LambdaRank, it efficiently learns to rank items in a way that directly improves metrics like NDCG. This post unpacks how LambdaMART works, why it’s effective, and how it fits into modern ranking architectures, especially when integrated with tools like LightGBM or platforms like Shaped.Wed, 30 Jul 2025 00:00:00 GMTIRhttps://tullie.ai/blog/average-popularity-are-your-recommendations-just-chasing-trends/https://tullie.ai/blog/average-popularity-are-your-recommendations-just-chasing-trends/Relevance metrics like NDCG and Precision@K are crucial for evaluating recommendation systems, but they don’t tell the full story. Two systems can perform similarly on these scores while exhibiting drastically different behaviors, one favoring only popular hits, the other surfacing more personalized or niche content. This is where Average Popularity @ K comes in. It quantifies the popularity bias of a model’s recommendations and helps diagnose whether it’s truly personalizing or simply echoing what’s already trending. Used alongside relevance metrics, it offers critical insight into model behavior and helps teams strike the right balance between accuracy and discovery.Thu, 24 Jul 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/decoding-timestamps-time-based-feature-engineering-for-search-recs/https://tullie.ai/blog/decoding-timestamps-time-based-feature-engineering-for-search-recs/Timestamps hold far more value than just marking when an event occurred, they encode powerful signals like recency, seasonality, user lifecycle, and content freshness that can significantly boost the performance of recommendation and search systems. But unlocking their potential requires careful feature engineering: time zone normalization, cyclical feature extraction, time-based calculations relative to “now,” and smart handling of missing values. This article breaks down best practices for transforming raw timestamp data into meaningful model features and highlights how platforms like Shaped simplify this process by automating temporal feature engineering, ensuring these critical signals are seamlessly incorporated into your ML models.Thu, 24 Jul 2025 00:00:00 GMTIRhttps://tullie.ai/blog/goodreads-datasets-powering-book-recommendations-and-research/https://tullie.ai/blog/goodreads-datasets-powering-book-recommendations-and-research/The GoodReads datasets are a foundational resource for building and evaluating book recommendation systems. They combine explicit ratings, implicit feedback (like user shelves), rich textual reviews, and detailed metadata, making them ideal for hybrid models that mix collaborative filtering with NLP. While the datasets vary in scope and format, they enable research into social influence, genre dynamics, and reader preferences at scale. Despite challenges like sparsity and ethical data handling, GoodReads remains one of the most valuable open datasets for exploring advanced recommendation strategies in the literary domain.Tue, 22 Jul 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/peering-inside-the-black-box-leveraging-user-item-embeddings/https://tullie.ai/blog/peering-inside-the-black-box-leveraging-user-item-embeddings/Embeddings are central to personalized recommendations: dense vector representations of users and items that capture behavioral patterns and semantic relationships.Fri, 18 Jul 2025 00:00:00 GMTIRhttps://tullie.ai/blog/dlrm-feature-interactions-ranking/https://tullie.ai/blog/dlrm-feature-interactions-ranking/Deep Learning Recommendation Models (DLRMs) like Wide & Deep, DeepFM, DCN, and MaskNet have become essential tools for pointwise ranking in recommendation systems, where the goal is to predict the likelihood of user-item interactions such as clicks or conversions. These models excel at capturing complex feature interactions across sparse, high-cardinality data by combining embedding layers with neural networks and specialized interaction mechanisms. This post breaks down how they work, why feature interactions matter, and how platforms like Shaped simplify building and deploying them for high-accuracy personalization.Thu, 17 Jul 2025 00:00:00 GMTIRhttps://tullie.ai/blog/pinrec-generative-retrieval/https://tullie.ai/blog/pinrec-generative-retrieval/"TL;DR: Pinterest's PinRec paper details a major leap in industrial-scale generative retrieval. By introducing "outcome-conditioned generation" to steer recommendations and "windowed multi-token generation" for efficiency, they built a transformer-based system that...Wed, 16 Jul 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/categorical-features-the-backbone-of-search-recs-engineering/https://tullie.ai/blog/categorical-features-the-backbone-of-search-recs-engineering/Categorical features like category, brand, and user ID are essential to search and recommendation systems, but transforming them into meaningful signals for machine learning is often more complex than it appears. This post explains how to handle categorical data effectively, from basic encoding strategies like one-hot and label encoding to deep learning approaches like embeddings. It covers challenges like high cardinality, null values, and feature consistency across systems, and shows how platforms like Shaped streamline this process by automating encoding, managing embeddings, and integrating categorical features directly into real-time relevance models.Tue, 15 Jul 2025 00:00:00 GMTIRhttps://tullie.ai/blog/gowalla-dataset-understanding-location-check-ins-social-ties-and-mobility-patterns/https://tullie.ai/blog/gowalla-dataset-understanding-location-check-ins-social-ties-and-mobility-patterns/The Gowalla dataset, a historical benchmark from the now-defunct location-based social network, offers rich check-in and social graph data that has powered foundational research in Point-of-Interest (POI) recommendations, human mobility modeling, and social influence on real-world behavior. Despite its age, Gowalla remains valuable for studying how time, geography, and social context shape user activity. This post explores its structure, use cases, limitations, and how to leverage it with Shaped to build context-aware recommendation models.Mon, 14 Jul 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/catalog-coverage-are-your-recommendations-exploring-your-whole-inventory/https://tullie.ai/blog/catalog-coverage-are-your-recommendations-exploring-your-whole-inventory/While traditional recommendation metrics focus on individual user experience, Catalog Coverage measures the breadth of a system’s recommendations across its entire inventory i.e how much of the catalog gets shown to anyone at all. It’s a valuable diagnostic for spotting over-reliance on popular items and uncovering long-tail neglect, but it ignores relevance and personalization. At Shaped, we treat coverage as a secondary signal, useful for monitoring systemic diversity, but never at the expense of delivering high-quality, personalized results.Fri, 11 Jul 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/content-based-filtering-explained-recommending-based-on-what-you-like/https://tullie.ai/blog/content-based-filtering-explained-recommending-based-on-what-you-like/Content-Based Filtering (CBF) is one of the fundamental approaches to building recommendation systems. Rather than relying on the preferences of similar users, CBF focuses on the characteristics of the items a user has engaged with to suggest others with similar attributes, whether textual, visual, structured, or audio-based. This article introduces how CBF works, its evolution from simple keyword matching to the use of modern embedding models, and the challenges involved in implementing it effectively. It also outlines different design patterns supported by Shaped for applying CBF in practice.Tue, 08 Jul 2025 00:00:00 GMTIRhttps://tullie.ai/blog/movielens-dataset-the-essential-benchmark-for-recommender-systems/https://tullie.ai/blog/movielens-dataset-the-essential-benchmark-for-recommender-systems/The MovieLens dataset is one of the most widely used benchmarks in recommender systems, offering real-world, explicit feedback data for evaluating collaborative filtering, content-based, and hybrid recommendation models. This article explores why MovieLens remains a gold standard, detailing its structure (ratings, metadata, tags), available versions, and common use cases. It also highlights challenges like data sparsity and cold start, and shows how to connect MovieLens to Shaped to quickly prototype and train recommendation models using real interaction data enriched with movie attributes.Wed, 02 Jul 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/from-zero-to-relevant-solving-the-cold-start-user-problem/https://tullie.ai/blog/from-zero-to-relevant-solving-the-cold-start-user-problem/New or anonymous users often face irrelevant, generic content, hurting engagement from the very first visit. This article explores the cold start user problem in personalization and search systems, outlining common strategies like global popularity lists, rule-based segments, onboarding surveys, and contextual inference. It highlights the challenges each approach presents and why effectively using even limited real-time context or early in-session behavior is key to delivering relevance from the start.Fri, 27 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/last-fm-datasets-unlocking-music-recommendations-through-listening-history-and-social-connections/https://tullie.ai/blog/last-fm-datasets-unlocking-music-recommendations-through-listening-history-and-social-connections/The article explores the significance of Last.fm datasets in developing music recommendation systems, highlighting their value as benchmarks for modeling implicit feedback, sequential listening behavior, and social influence. It breaks down what’s included in these datasets (such as user listening history, social graphs, and tags) and why they matter for music personalization research. It also walks through how teams can bring these datasets into Shaped to build real-time ranking models, covering schema setup, event ingestion, and optional use of tags or social data, demonstrating how Shaped makes it easy to prototype and productionize music recommenders using this rich, real-world data.Fri, 27 Jun 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/privacy-first-personalization/https://tullie.ai/blog/privacy-first-personalization/Personalization has become a standard expectation across digital experiences, from streaming platforms to e-commerce sites. However, as consumers become increasingly aware of how their data is used, and as regulations tighten, businesses face a new challenge: delivering relevance without compromising trust.Tue, 24 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/see-the-bigger-picture-image-feature-engineering-for-search-recs/https://tullie.ai/blog/see-the-bigger-picture-image-feature-engineering-for-search-recs/In visually rich digital environments, text and tags alone often fall short in powering relevant search and recommendations. This article explores how visual feature engineering, extracting embeddings from images using models like CLIP or ViT, unlocks deeper relevance by capturing visual nuance, style, and cross-modal meaning. While traditional computer vision pipelines are complex and resource-intensive, Shaped streamlines the entire process: ingesting image URLs, generating embeddings with advanced models, and integrating them into ranking APIs, all without requiring custom infrastructure. Whether automatically leveraging visuals or specifying your own Hugging Face model, Shaped makes it simple to activate image data for AI-powered personalization.Mon, 23 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/how-youtubes-algorithm-works/https://tullie.ai/blog/how-youtubes-algorithm-works/YouTube’s recommendation engine combines large-scale data processing, real-time feedback loops, and multi-objective optimization to deliver highly personalized video suggestions that prioritize both engagement and satisfaction. This post breaks down how the system works, from candidate generation to safeguards, and offers actionable lessons for building adaptable, responsible recommendation systems of your own.Thu, 19 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/mrr-how-quickly-do-users-find-the-first-relevant-item/https://tullie.ai/blog/mrr-how-quickly-do-users-find-the-first-relevant-item/Mean Reciprocal Rank (MRR) is a metric that captures how quickly a user finds the first relevant item in a ranked list, making it especially valuable for tasks like known-item search or question answering where just one good result matters. This article introduces the concept of MRR, explains how it's calculated, compares it to other ranking metrics like NDCG and Hit Rate, and explores when it’s most useful (and when it’s not). It also outlines how Shaped incorporates MRR into its broader evaluation suite to balance speed of discovery with overall relevance.Thu, 19 Jun 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/mastering-cold-start-challenges/https://tullie.ai/blog/mastering-cold-start-challenges/Cold start challenges can derail personalization efforts by making it difficult to deliver relevant experiences for new users, items, or markets. This post explores proven strategies and modern system architectures — including modular, AI-native platforms like Shaped — that help teams overcome cold start and personalize from day one.Wed, 18 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/explainable-personalization/https://tullie.ai/blog/explainable-personalization/Personalization helps users discover the right content, products, or experiences, but when it happens without explanation, it can feel invasive, confusing, or even manipulative. As algorithms play a larger role in shaping what we see, hear, and buy, users are beginning to ask a simple question: *Why am I seeing this?* That question isn’t just philosophical. It reflects a growing demand for transparency, control, and trust in algorithmic systems. Whether you're building a recommendation engine, a personalized feed, or a product ranking feature, explainability is becoming essential. It reassures users, supports compliance, and helps teams understand and improve their models.Tue, 17 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/matrix-factorization-the-bedrock-of-collaborative-filtering-recommendations/https://tullie.ai/blog/matrix-factorization-the-bedrock-of-collaborative-filtering-recommendations/Matrix Factorization (MF) has long been a foundational technique in collaborative filtering for recommendation systems. It works by learning latent factors that represent hidden preferences of users and characteristics of items, allowing it to predict unknown interactions. This article explains how MF decomposes the sparse user-item interaction matrix into two lower-dimensional matrices, and dives into popular optimization methods like Stochastic Gradient Descent (SGD) and Alternating Least Squares (ALS), including how ALS adapts to implicit feedback with confidence weighting. The post covers enhancements like user/item biases, practical challenges like cold-start, and how MF compares to neighborhood and deep learning approaches. Finally, it shows how platforms like Shaped let teams deploy ALS-based recommendations declaratively, without building pipelines from scratch.Tue, 17 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/modular-ai/https://tullie.ai/blog/modular-ai/As user expectations rise and product surfaces multiply, personalization systems are under more pressure than ever. But many teams still operate with rigid, monolithic architectures that make every change slow, risky, and expensive. Updating a ranking strategy, testing a new model, or even adding a new content source can require changes across the entire stack.Mon, 16 Jun 2025 00:00:00 GMTEnghttps://tullie.ai/blog/ndcg-graded-relevance/https://tullie.ai/blog/ndcg-graded-relevance/How do you know if your ranking model is getting the order right, not just retrieving the right items? This post introduces NDCG, a powerful metric that accounts for both how relevant each item is and where it appears in the ranked list, enabling a more nuanced evaluation of recommendation and search quality, especially when relevance varies across results.Thu, 12 Jun 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/10-best-practices-in-data-ingestion/https://tullie.ai/blog/10-best-practices-in-data-ingestion/Every real-time dashboard, machine learning model, and personalized user experience depends on one foundational layer: data ingestion. It's the first step in any modern data pipeline, responsible for collecting, validating, and delivering data from source systems into downstream platforms where it can be analyzed, modeled, or acted upon.Wed, 11 Jun 2025 00:00:00 GMTEnghttps://tullie.ai/blog/unlock-text-data-nlp-feature-engineering-for-search-recs/https://tullie.ai/blog/unlock-text-data-nlp-feature-engineering-for-search-recs/Keyword matching and interaction history aren’t enough for modern relevance. Language data, like product descriptions, search queries, and user reviews, holds rich signals that drive deeper personalization. But turning text into model-ready features requires complex NLP pipelines, model selection, infrastructure, and ongoing maintenance. Shaped automates all of this. With built-in language understanding and Hugging Face model integration, teams can tap into the full power of semantic signals, without building or managing an NLP stack.Wed, 11 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/monolithic-vs-modular-ai-architecture/https://tullie.ai/blog/monolithic-vs-modular-ai-architecture/The distinction between monolithic and modular approaches is a key consideration of AI-native architectures. Monolithic architectures package all AI components into a single system, while modular approaches split functionality into independent services. For AI personalization systems — architectures explicitly designed for machine learning workloads, real-time data processing, and dynamic recommendation engines—this choice shapes everything from development velocity to long-term scalability.Mon, 09 Jun 2025 00:00:00 GMTEnghttps://tullie.ai/blog/how-to-unify-data-ecosystems/https://tullie.ai/blog/how-to-unify-data-ecosystems/Companies that excel at personalization generate 40% more revenue than their competitors. Yet, most struggle with a basic problem: fragmented data ecosystems. When customer information sits isolated in different systems, you miss the complete picture needed for truly effective personalization.Sat, 07 Jun 2025 00:00:00 GMTEnghttps://tullie.ai/blog/ai-powered-recommendation-engines/https://tullie.ai/blog/ai-powered-recommendation-engines/Recommendation engines have become an essential part of the online shopping experience, enabling businesses to deliver personalized suggestions that resonate with customers. By analyzing user data and behavior, these systems offer tailored product recommendations, helping users discover what they’re most likely to enjoy or purchase next. Recommendation systems powered by artificial intelligence (AI) are at the forefront of this shift. As the AI-based recommendation system market is [expected to grow](https://www.thebusinessresearchcompany.com/report/ai-based-recommendation-system-global-market-report) from $2.44 billion in 2025 to $3.62 billion by 2029, it is clear that the adoption of these systems is expanding rapidly across various industries, including e-commerce, healthcare, and digital advertising.Wed, 04 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/h-m-dataset-powering-personalized-fashion-recommendations-at-scale/https://tullie.ai/blog/h-m-dataset-powering-personalized-fashion-recommendations-at-scale/The H&M Personalized Fashion Recommendations dataset is a favorite in the ML community for testing large-scale, real-world recommendation systems. With millions of transactions and rich metadata, it offers a challenging benchmark for building personalized fashion experiences. In this post, we show how to connect the H&M dataset to Shaped, an AI-native relevance platform, to go beyond basic co-purchase signals. From implicit feedback and cold-start handling to hybrid ranking with item and user features, Shaped helps teams build smarter fashion recommenders, faster.Wed, 04 Jun 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/customer-data-platform/https://tullie.ai/blog/customer-data-platform/Effective personalization hinges on the ability to collect, manage, and analyze diverse streams of customer data. However, unlocking the full potential of this data requires sophisticated data mining techniques and robust infrastructure to transform raw data into actionable insights.Tue, 03 Jun 2025 00:00:00 GMTEnghttps://tullie.ai/blog/feed-ranking-systems/https://tullie.ai/blog/feed-ranking-systems/Recommendation systems are fundamental to modern digital platforms. They curate the vast content users encounter daily on social media, streaming services, and e-commerce sites. These systems engage users by providing personalized experiences that align with their interests and behaviors. However, managing and optimizing these systems for platforms with over 1 billion members globally presents monumental challenges. The sheer scale necessitates sophisticated techniques not only in model architecture but also in efficient training, compression, and deployment to production.Tue, 03 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/multi-armed-bandits/https://tullie.ai/blog/multi-armed-bandits/Personalization has become the backbone of engaging user experiences across industries. But delivering smart personalization isn’t easy. Traditional approaches like A/B testing can be slow, rigid, and resource-intensive. They often force teams to pick one option at a time, missing opportunities to learn and adapt as user preferences shift quickly.Tue, 03 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/approximate-nearest-neighbors-algorithms/https://tullie.ai/blog/approximate-nearest-neighbors-algorithms/Finding the most relevant items from vast datasets is a fundamental challenge in modern machine learning applications. Whether you’re recommending movies on a streaming platform, suggesting products in an online store, or searching for similar images, the ability to quickly locate the nearest neighbor, or neighbors, to a given query point in high-dimensional spaces is critical. Traditional nearest neighbor search algorithms can identify the closest points by calculating exact distances, such as Euclidean distance, between vectors representing data points. But as datasets grow larger and more complex, especially with high-dimensional data like images, text embeddings, or user behavior patterns, exact search becomes prohibitively slow.Mon, 02 Jun 2025 00:00:00 GMTEngIRhttps://tullie.ai/blog/how-does-temu-work/https://tullie.ai/blog/how-does-temu-work/In just a few short years, Temu has evolved from a relatively unknown marketplace to one of the world's fastest-growing e-commerce platforms. Its rapid ascent has left industry watchers asking the same question: how did they do it? The answer lies in Temu’s strategic use of artificial intelligence to power engagement at every step of the user journey.Mon, 02 Jun 2025 00:00:00 GMTIRhttps://tullie.ai/blog/retrieval-augmented-generation-rag/https://tullie.ai/blog/retrieval-augmented-generation-rag/In a world where personalization and relevance are paramount, AI-driven systems often struggle to handle real-time data and maintain up-to-date information. Traditional models, while powerful, are limited by their reliance on static training data and their inability to adapt quickly to new or unstructured data. As organizations face knowledge-intensive tasks like answering complex queries, providing personalized recommendations, or generating content, the cost of maintaining large, constantly updated datasets becomes a significant barrier.Mon, 02 Jun 2025 00:00:00 GMTEngIRhttps://tullie.ai/blog/adaptrec-collaborative-filtering-llms/https://tullie.ai/blog/adaptrec-collaborative-filtering-llms/TL;DR: LLMs are powerful, but making them use collaborative filtering (CF) signals effectively for sequential recommendations is tricky. " Results show significant HR@1 improvements (7-18%) over traditional and other LLM-based methods, especially in few-shot scenarios.Fri, 30 May 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/how-amazon-masterminds-real-time-product-discovery/https://tullie.ai/blog/how-amazon-masterminds-real-time-product-discovery/While many retailers focus on making search faster or more accurate, Amazon has mastered the art of guiding users beyond the search bar. The scale of Amazon’s success is a testament to this mastery. According to [Statista](https://www.statista.com/statistics/273963/quarterly-revenue-of-amazoncom/), during the first quarter of 2024, Amazon generated total net sales of over $143 billion, surpassing the $127 billion from the same quarter in 2023. This relentless growth is powered by Amazon’s sophisticated approach to real-time product discovery and personalization.Fri, 30 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/relevancy-precision-and-recall/https://tullie.ai/blog/relevancy-precision-and-recall/It’s not enough to just serve up suggestions. You need to serve the *right* suggestions. That’s where understanding performance metrics like precision and recall comes in. These evaluation metrics help you measure how well your machine learning (ML) model identifies relevant results and balances the tricky trade-off between minimizing false positives and false negatives. Precision and recall shape how users experience your product and influence tangible business outcomes. For example, a high precision score means your system makes fewer false alarms by minimizing irrelevant recommendations, while high recall ensures you’re not missing out on positive cases that matter.Fri, 30 May 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/ai-powered-cross-selling-recommendations/https://tullie.ai/blog/ai-powered-cross-selling-recommendations/Cross-selling succeeds because it helps shoppers discover complementary products or add-ons they might not have considered, creating a more complete and satisfying purchase without becoming pushy. Customers benefit by finding exactly what they need, while retailers boost sales and maximize revenue from their existing customer base.Thu, 29 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/ab-testing-ranking-metrics/https://tullie.ai/blog/ab-testing-ranking-metrics/You’ve trained a model, optimized offline metrics, and picked a winner, but how do you know it’ll perform with real users? In this post, we explore why A/B testing is essential for validating personalization and ranking models in production. We cover key online metrics like CTR, CVR, and North Star Metrics, how to design statistically rigorous experiments, and how Shaped makes it easy to deploy, bucket, and measure real-world impact.Wed, 28 May 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/deep-learning-for-hyper-personalized-recommendations/https://tullie.ai/blog/deep-learning-for-hyper-personalized-recommendations/Whether browsing for products, discovering new content, or navigating a website, users now demand a personalized experience tailored to their unique preferences and behaviors. However, traditional recommendation systems, often built on simple rule-based or content-based filtering systems, struggle to deliver the dynamic, context-aware experiences users crave.Tue, 27 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/golden-tests-in-ai/https://tullie.ai/blog/golden-tests-in-ai/For teams building AI-driven experiences, especially those delivering real-time recommendations, speed is everything. Whether you're personalizing a homepage feed or updating product rankings, models must continually evolve to stay relevant. But that velocity comes with risk. Even minor changes to a model or pipeline can have unexpected consequences once deployed. A tweak meant to boost click-through rates might unintentionally bury high-converting items.Mon, 26 May 2025 00:00:00 GMTEnghttps://tullie.ai/blog/bridging-worlds-training-language-models-on-user-behavior-for-smarter-recommendations/https://tullie.ai/blog/bridging-worlds-training-language-models-on-user-behavior-for-smarter-recommendations/Traditional recommendation models face a tradeoff: language models excel at understanding item semantics, while collaborative filtering shines at capturing behavioral patterns. But what if you could combine both? In this post, we explore a new generation of hybrid techniques, like the beeFormer framework, that fine-tune pre-trained language models using user interaction data. The result: smarter, cold-start-ready embeddings that understand both meaning and behavior. We break down how this works, why it matters, and how platforms like Shaped make it easy to put these powerful models into production.Fri, 23 May 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/evaluation-metrics-for-search-and-recommendation-systems/https://tullie.ai/blog/evaluation-metrics-for-search-and-recommendation-systems/Search and recommendation systems power everything from e-commerce product discovery to streaming content suggestions. Without clear, effective metrics, it is impossible to measure how well they perform or identify areas for improvement.Thu, 22 May 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/modern-ranking-models/https://tullie.ai/blog/modern-ranking-models/People today are inundated with choices, whether browsing products, searching for information, or discovering new content. The challenge for businesses is not just to present options, but to ensure the most relevant, engaging, and valuable items rise to the top. This is where ranking models come into play. These sophisticated algorithms power the search results you see, the recommendations you receive, and the content you’re most likely to click, watch, or buy. Ranking models are at the heart of personalization and discovery in industries ranging from e-commerce and media to online marketplaces. We’ll demystify ranking models, explore their key components, and outline best practices for implementing them across various use cases.Mon, 19 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/collaborative-filtering/https://tullie.ai/blog/collaborative-filtering/Have you ever wondered how Netflix seems to know what to suggest next or how Amazon always recommends products you'll likely buy? This isn’t by chance. It's powered by recommendation engines. The global recommendation engine market was [valued at $5.43 billion in 2023](https://www.kingsresearch.com/recommendation-engine-market-1945) and is expected to grow rapidly, reaching $74.24 billion by 2031.Sun, 18 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/vector-search-explained/https://tullie.ai/blog/vector-search-explained/Search is undergoing a quiet transformation. As users expect instant, relevant results, whether shopping online, exploring a streaming platform, or using an AI assistant, traditional keyword search is no longer enough. Leading companies like Netflix, Amazon, and Spotify already use a different approach behind the scenes: vector search. We’ll explore how vector search powers today’s most advanced discovery and recommendation systems, looking at how it works, where it fits into modern AI infrastructure, and why it’s becoming a cornerstone of user experience across industries.Thu, 15 May 2025 00:00:00 GMTEngIRhttps://tullie.ai/blog/tweedie-regression-video-recommendations-tubi/https://tullie.ai/blog/tweedie-regression-video-recommendations-tubi/TL;DR: Tubi boosted VOD revenue (+0.4%) and watch time (+0.15%) by ditching weighted LogLoss for CTR and instead using Tweedie Regression to directly predict user watch time. Their paper shows Tweedie loss better models the zero-inflated, skewed nature of watch time data, leading to better alignment with core business goals, even with a slight dip in a simpler conversion metric.Wed, 14 May 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/netflix-personalization-workshop-2025/https://tullie.ai/blog/netflix-personalization-workshop-2025/The Shaped team was thrilled to be at the 2025 Netflix Personalization, Recommendations & Search workshop last week! This event, first held by Netflix in 2016, is one of our highlights on the AI recommendation & search calendar. The day was packed with insightful talks from academic and industry leaders, all tackling the fast-paced evolution of AI-driven user experiences. While Large Foundation Models (LFMs) and Generative AI were, as expected, major topics, the conversations dug deep into real-world applications, innovative architectures, and the changing face of AI product development. Here’s our summary of the keynotes and insights that stood out.Tue, 13 May 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/wayfair-pinterest/https://tullie.ai/blog/wayfair-pinterest/Personalized discovery has become essential for digital platforms aiming to engage users effectively. Today’s consumers expect experiences that reflect their unique tastes, especially when browsing visually driven categories such as home goods or lifestyle content.Tue, 13 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/genret-semantic-tokenization/https://tullie.ai/blog/genret-semantic-tokenization/Generative retrieval is emerging as a transformative approach to document retrieval, leveraging generative language models (LMs) to directly produce ranked lists of document identifiers (docids) for user queries. The paper "Learning to Tokenize for Generative Retrieval"...Mon, 12 May 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/two-tower-recommendation-models/https://tullie.ai/blog/two-tower-recommendation-models/The Two-Tower model is a foundational architecture for large-scale recommendation systems, built to efficiently retrieve relevant items from massive catalogs. By learning separate embeddings for users and items, it enables fast candidate generation via approximate nearest neighbor search—critical for real-time personalization. This article breaks down how the model works, why it scales, and where it fits in modern recsys stacks, highlighting its strengths, limitations, and role alongside ranking and graph-based approaches.Fri, 09 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/criteo-dataset-tackling-large-scale-click-through-rate-prediction/https://tullie.ai/blog/criteo-dataset-tackling-large-scale-click-through-rate-prediction/Click-through rate (CTR) prediction is central to modern advertising and recommendation systems, and the Criteo dataset has become the de facto benchmark for advancing this task at industrial scale. With hundreds of millions to billions of rows and a blend of dense numerical and sparse categorical features, it poses unique modeling and computational challenges. This article unpacks the dataset’s structure, scale, and role in driving innovations like embedding techniques and hybrid model architectures—offering a clear lens into why Criteo remains a crucial resource for anyone building large-scale machine learning systems.Thu, 08 May 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/sequential-models-for-recommendations/https://tullie.ai/blog/sequential-models-for-recommendations/In a world where user behavior changes by the minute, traditional recommendation systems fall short. Sequential recommendation models offer a powerful upgrade, capturing evolving intent by analyzing the order of interactions. This article breaks down the evolution of these models, from simple N-Grams to advanced Transformers and Generative Recommenders like HSTU. It also explores the real-world challenges of deploying them and how platforms like Shaped make cutting-edge sequential modeling accessible, scalable, and production-ready.Tue, 06 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/soar-orthogonality-amplified-ann-indexing/https://tullie.ai/blog/soar-orthogonality-amplified-ann-indexing/According to recent research "SOAR: Improved Indexing for Approximate Nearest Neighbor Search" by Google researchers, published in NeurIPS 2023, SOAR (Spilling with Orthogonality-Amplified Residuals) introduces a novel data indexing technique for approximate nearest neighbor...Mon, 05 May 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/how-to-build-a-killer-for-you-feed/https://tullie.ai/blog/how-to-build-a-killer-for-you-feed/The “For You” feed has become the gold standard of personalized digital experiences—but behind the magic lies serious technical complexity. From wrangling massive datasets to training cutting-edge ML models and serving results in real time, building a high-quality feed from scratch demands deep expertise and infrastructure. This post breaks down the full journey: what it takes to deliver a truly personalized feed, the common pain points at each stage, and how to think strategically about solving them—whether you're just getting started or scaling an existing system.Fri, 02 May 2025 00:00:00 GMTIRhttps://tullie.ai/blog/empathetic-conversational-recommenders/https://tullie.ai/blog/empathetic-conversational-recommenders/Conversational recommender systems (CRSs) have made significant strides in eliciting user preferences through multi-turn dialogues, but they often overlook the emotional aspects of user interactions. , presenting at the 18th ACM Conference on Recommender Systems (RecSys '24),...Tue, 29 Apr 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/beyond-retrieval-optimizing-relevance-with-reranking/https://tullie.ai/blog/beyond-retrieval-optimizing-relevance-with-reranking/Retrieving a strong list of candidate items is just the first step—the real challenge is ranking them in the most relevant, personalized order for each user and goal. This post explores how reranking transforms basic search results or recommendations into truly optimized experiences, the technical hurdles of building high-performance reranking systems, and why mastering reranking is key to delivering better engagement, clicks, and conversions.Mon, 28 Apr 2025 00:00:00 GMTIRhttps://tullie.ai/blog/precision-at-k-ranking/https://tullie.ai/blog/precision-at-k-ranking/Is your recommender system truly hitting the mark? Imagine a user binging blockbusters like Avengers and Top Gun—will they click on Love Actually or John Wick next? This article breaks down Precision@K, the go-to metric for judging how many of your top K recommendations are actually relevant. With clear, intuitive examples and a sharp look at where the metric excels—and where it doesn’t—you’ll get a fast, practical understanding of how to measure recommendation quality where it matters most: the top of the list.Fri, 25 Apr 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/cross-encoder-rediscovers-a-semantic-variant-of-bm25/https://tullie.ai/blog/cross-encoder-rediscovers-a-semantic-variant-of-bm25/We stand in awe of modern neural ranking models. Transformers like BERT, fine-tuned as cross-encoders, achieve state-of-the-art results on information retrieval leaderboards. They process a query and document together, capturing incredibly nuanced semantic relationships, far surpassing traditional methods like BM25. But how do they do it? We often treat them as powerful black boxes: feed them data, get amazing results, but shrug when asked about the internal logic.Thu, 24 Apr 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/one-embedding-to-rule-them-all/https://tullie.ai/blog/one-embedding-to-rule-them-all/Pinterest’s OmniSearchSage represents a major step forward in unified semantic search. By extending the two-tower model into a multi-task, multi-entity framework, it enables a single query embedding to power retrieval across pins, products, and related queries. The system integrates GenAI captions, user-curated board metadata, and behavioral signals to overcome sparse content, while maintaining compatibility with legacy models like PinSage. Deployed at massive scale, OmniSearchSage delivers strong gains in search fulfillment, ad performance, and downstream tasks, showcasing a pragmatic and scalable approach to representation learning in production.Tue, 22 Apr 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/jagged-flash-attention-optimization/https://tullie.ai/blog/jagged-flash-attention-optimization/Meta researchers have introduced Jagged Flash Attention, a novel technique that significantly enhances the performance and scalability of large-scale recommendation systems. By combining jagged tensors with flash attention, this innovation achieves up to 9× speedup and 22×...Tue, 18 Mar 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/beyond-relevance-optimizing-for-multiple-objectives-in-search-and-recommendations/https://tullie.ai/blog/beyond-relevance-optimizing-for-multiple-objectives-in-search-and-recommendations/Building effective recommendation and search systems means going beyond simply predicting relevance. Modern users expect personalized experiences that cater to a wide range of needs and preferences, and businesses need systems that align with their overarching goals. This requires optimizing for multiple objectives simultaneously – a complex challenge that demands a nuanced approach. This post explores the concept of value modeling and multi-objective optimization (MOO), explaining how these techniques enable the development of more sophisticated and valuable recommendation and search experiences.Wed, 05 Mar 2025 00:00:00 GMTIRhttps://tullie.ai/blog/beyond-dot-products-retrieval-with-learned-similarities/https://tullie.ai/blog/beyond-dot-products-retrieval-with-learned-similarities/This paper, by Bailu Ding (Microsoft) and Jiaqi Zhai (Meta), which is in the proceedings of the WWW '25 conference, proposes a novel approach called Mixture of Logits (MoL) that offers a generalized interface for learned similarity functions. It not only achieves state-of-the-art results across recommendation systems and question answering but also demonstrates significant latency improvements, potentially reshaping the landscape of vector databases.Thu, 27 Feb 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/ab-testing-statistical-power/https://tullie.ai/blog/ab-testing-statistical-power/A/B testing metrics are crucial for evaluating recommender systems and guiding decision-making for digital platforms, as demonstrated by ShareChat in "Powerful A/B-Testing Metrics and Where to Find Them" . The challenge lies in identifying metrics with high statistical power...Thu, 20 Feb 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/multimodal-alignment-for-recommendations/https://tullie.ai/blog/multimodal-alignment-for-recommendations/In the rapidly evolving landscape of recommendation systems, an approach called AlignRec, in the paper "AlignRec: Aligning and Training in Multimodal Recommendations" , is addressing the critical challenge of misalignment in multimodal recommendations. , in a recent CIKM '24...Thu, 13 Feb 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/masknet-ctr-ranking-innovation/https://tullie.ai/blog/masknet-ctr-ranking-innovation/In 2021, before the AI boom sparked by ChatGPT, Sina Weibo Corp researchers introduced MaskNet, "MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask", at DLP-KDD, ACM,Singapore. This feature-wise multiplication approach to...Tue, 11 Feb 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/data-splitting-can-make-or-break-your-recommender-system/https://tullie.ai/blog/data-splitting-can-make-or-break-your-recommender-system/How frustrating it is if a dessert you made at home simply refuses to taste as good if made at a friend’s place? A good recipe should work even if you change kitchens. This should especially be true in scientific experiments. Using the same data on the same models, you must...Wed, 05 Feb 2025 00:00:00 GMTEvalshttps://tullie.ai/blog/embsum-llm-powered-content-recommendations/https://tullie.ai/blog/embsum-llm-powered-content-recommendations/Content-based recommendation systems are essential for delivering personalized content in the digital world. EmbSum leverages the summarization capabilities of Large Language Models (LLMs) to transform content recommendations. By enabling offline pre-computation of user and...Wed, 29 Jan 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/titans-neural-memory/https://tullie.ai/blog/titans-neural-memory/Google Research's latest paper in December 2024 , "Titans: Learning to Memorize at Test Time" introduces a groundbreaking neural long-term memory module that learns to memorize historical context at test time, potentially revolutionizing how AI models handle extended...Fri, 17 Jan 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/explainable-job-recommendations/https://tullie.ai/blog/explainable-job-recommendations/Imagine a world where AI not only finds the perfect job for a candidate but also helps recruiters identify top talent and enables organizations to make smarter hiring decisions—all before anyone says a word. While this may sound like science fiction, AI-powered job matching...Wed, 15 Jan 2025 00:00:00 GMTResearchIRhttps://tullie.ai/blog/cosine-similarity-not-the-silver-bullet-we-thought-it-was/https://tullie.ai/blog/cosine-similarity-not-the-silver-bullet-we-thought-it-was/In the world of machine learning and data science, cosine similarity has long been a go-to metric for measuring the semantic similarity between high-dimensional objects. However, a new study by researchers at Netflix and Cornell University challenges our understanding of this...Mon, 13 Jan 2025 00:00:00 GMTResearchhttps://tullie.ai/blog/improving-recommendations-by-calibrating-for-user-interests/https://tullie.ai/blog/improving-recommendations-by-calibrating-for-user-interests/"Sometimes I make the mistake of listening to some new pop songs back-to-back on my music app for a while and soon after most of my recommendations are just the latest pop. This also applies to liking social media posts, streaming movies, and even shopping sites. Things go...Thu, 19 Dec 2024 00:00:00 GMTResearchIRhttps://tullie.ai/blog/vector-search-lucene-is-all-you-need/https://tullie.ai/blog/vector-search-lucene-is-all-you-need/Recent research is challenging long-held assumptions about AI-powered search, questioning whether dedicated vector stores are truly necessary. As AI and search technologies continue to evolve, organizations are searching for solutions that balance efficiency, cost, and...Mon, 16 Dec 2024 00:00:00 GMTResearchhttps://tullie.ai/blog/pinsage-web-scale-recommenders/https://tullie.ai/blog/pinsage-web-scale-recommenders/In this article, we'll take a deep dive into PinSage, a state-of-the-art GCN framework developed at Pinterest for learning high-quality embeddings of nodes in massive, billion-scale graphs. Through a novel combination of techniques spanning sampling, dynamic graph...Thu, 14 Nov 2024 00:00:00 GMTResearchIRhttps://tullie.ai/blog/effective-caching-strategies-for-recommender-systems/https://tullie.ai/blog/effective-caching-strategies-for-recommender-systems/In the realm of recommendation systems, performance and speed are crucial. This post will guide you through advanced caching strategies designed to optimize data retrieval and minimize latency in your recommendation system. Implementing these techniques can greatly improve...Thu, 07 Nov 2024 00:00:00 GMTEnghttps://tullie.ai/blog/deep-reinforcement-learning-recommenders/https://tullie.ai/blog/deep-reinforcement-learning-recommenders/This article presents a technical exploration of deep reinforcement learning (DRL) in recommender systems, focusing on the latest methodologies, architectures, and algorithms. We provide a detailed survey of how DRL is applied to overcome the challenges of traditional...Wed, 16 Oct 2024 00:00:00 GMTResearchIRhttps://tullie.ai/blog/recommender-system-family-tree-gnn/https://tullie.ai/blog/recommender-system-family-tree-gnn/In this article, we will explore the world of GNNs in the context of recommender systems, delving into their unique advantages and the various ways they enhance the recommendation process. From collaborative filtering to session-based recommendation and knowledge-aware...Wed, 16 Oct 2024 00:00:00 GMTIRhttps://tullie.ai/blog/learning-to-rank-for-recommender-systems/https://tullie.ai/blog/learning-to-rank-for-recommender-systems/In this practical guide, we dive deep into the world of learning to rank for recommender systems, exploring its fundamental concepts, key benefits, and step-by-step implementation process. Whether you're new to the field or looking to refine your existing knowledge, this...Wed, 25 Sep 2024 00:00:00 GMTIRhttps://tullie.ai/blog/exploration-in-recommender-systems/https://tullie.ai/blog/exploration-in-recommender-systems/Researchers at Google DeepMind recently published an insightful paper that delves into the long-term benefits of exploration within recommendation platforms. They argue that while short-term metrics might not immediately reflect the advantages, exploration can significantly...Thu, 05 Sep 2024 00:00:00 GMTResearchIRhttps://tullie.ai/blog/breaking-down-toolformer/https://tullie.ai/blog/breaking-down-toolformer/Take a look at our discussion on Toolformer — Meta AI's recent approach to fusing large language models (LLM) with external APIs. This might be the start of a new programming paradigm that combines zero-shot machine-learning methodology with traditional software interfaces.Tue, 16 Jul 2024 00:00:00 GMTResearchhttps://tullie.ai/blog/offline-vs-online-evaluation/https://tullie.ai/blog/offline-vs-online-evaluation/Evaluating recommendation models is notoriously difficult and there is rarely a silver bullet approach. This article walks through different model evaluation methods, the common pitfalls when evaluating recommendation models, and how to avoid them.Tue, 09 Jul 2024 00:00:00 GMTEvalshttps://tullie.ai/blog/chatgpt-moment-for-recommendations/https://tullie.ai/blog/chatgpt-moment-for-recommendations/Researchers at Meta recently published a ground-breaking paper that combines the technology behind ChatGPT with Recommender Systems. They show they can scale these models up to 1.5 trillion parameters and demonstrate a 12.4% increase in topline metrics in production A/B tests. We dive into the details below.Wed, 05 Jun 2024 00:00:00 GMTResearchIRhttps://tullie.ai/blog/a-technical-intro-to-embeddings/https://tullie.ai/blog/a-technical-intro-to-embeddings/This blog takes a deeper look into how the embeddings that fuel all modern AI powered systems are generated, and the decisions made when creating these. If you want to know how all your favourite AI's understand your unstructured data, read on!Tue, 16 Apr 2024 00:00:00 GMTIRhttps://tullie.ai/blog/embracing-embeddings/https://tullie.ai/blog/embracing-embeddings/This blog delves into the transformative potential behind embedding techniques in data science over previous traditional methods, and how companies can harness this leap in order to unlock the true power of all of their data.Tue, 28 Nov 2023 00:00:00 GMTIRhttps://tullie.ai/blog/how-much-data-do-i-need-for-a-recommendation-system-part-2/https://tullie.ai/blog/how-much-data-do-i-need-for-a-recommendation-system-part-2/If you’re interested in recommendation systems but not sure whether you have enough data this blog post is for you! If you haven't read Part 1, a link is below.Tue, 07 Nov 2023 00:00:00 GMTIRhttps://tullie.ai/blog/rag-for-recsys-a-magic-formula/https://tullie.ai/blog/rag-for-recsys-a-magic-formula/The blog explores "Retrieval-Augmented Generation" (RAG), which melds information retrieval with language generation, to evaluate its promise for recommendations.Mon, 16 Oct 2023 00:00:00 GMTIRhttps://tullie.ai/blog/threads-recommendation-system/https://tullie.ai/blog/threads-recommendation-system/What Tiktok did to personalize short-form video, Threads will do for the digital town square.Thu, 10 Aug 2023 00:00:00 GMTIRhttps://tullie.ai/blog/exploring-benefits-of-llms-in-recsys/https://tullie.ai/blog/exploring-benefits-of-llms-in-recsys/This blog post explores the potential of using large language models in recommendation systems, optimal integration points and challenges in real-world applications such as training efficiency, inference latency and bias.Tue, 20 Jun 2023 00:00:00 GMTIRhttps://tullie.ai/blog/llms-a-paradigm-shift-in-recsys/https://tullie.ai/blog/llms-a-paradigm-shift-in-recsys/This blog post explores the potential of large language models (LLMs) as powerful recommendation systems, highlighting their ability to understand context and meaning to provide personalized suggestions, particularly focusing on their application in movie recommendations.Fri, 16 Jun 2023 00:00:00 GMTIRhttps://tullie.ai/blog/how-synthetic-data-is-used-to-train-machine-learning-models/https://tullie.ai/blog/how-synthetic-data-is-used-to-train-machine-learning-models/In this blog post we discuss the concept, advantages, and generation methods of synthetic data, and its increasing importance in training AI models, especially for overcoming biases, ensuring privacy, and reducing costs.Thu, 01 Jun 2023 00:00:00 GMTEnghttps://tullie.ai/blog/not-your-average-recsys-metrics-part-2-novelty/https://tullie.ai/blog/not-your-average-recsys-metrics-part-2-novelty/This blog post discusses the importance of novelty in recommendation systems and how it can improve user experience by providing diverse and unexpected suggestions.Mon, 08 May 2023 00:00:00 GMTEvalshttps://tullie.ai/blog/not-your-average-recsys-metrics-part-1-serendipity/https://tullie.ai/blog/not-your-average-recsys-metrics-part-1-serendipity/Let’s go beyond standard machine learning performance measurements and explore at how we can create Serendipity in a high performance recommendation systemFri, 28 Apr 2023 00:00:00 GMTEvalshttps://tullie.ai/blog/how-much-data-do-i-need-for-a-recommendation-system/https://tullie.ai/blog/how-much-data-do-i-need-for-a-recommendation-system/If you’re interested in recommendation systems but not sure whether you have enough data this blog post is for you!Tue, 25 Apr 2023 00:00:00 GMTIRhttps://tullie.ai/blog/twitters-open-source-algorithm/https://tullie.ai/blog/twitters-open-source-algorithm/X has recently unveiled its open-source recommendation algorithm, aiming to offer users greater transparency into the process through which the platform selects and organizes content for display on their timelines.Fri, 31 Mar 2023 00:00:00 GMTIRhttps://tullie.ai/blog/evaluating-recommendations-roc-auc/https://tullie.ai/blog/evaluating-recommendations-roc-auc/You probably have heard of terms like ROC, AUC, and Precision-recall, they show up in data science articles on Medium, machine learning tutorials, and academic papers are full of them. But why are they so important and what do they actually mean? Today we will dive into the...Tue, 28 Mar 2023 00:00:00 GMTEvalshttps://tullie.ai/blog/llama-democratizing-large-language-models/https://tullie.ai/blog/llama-democratizing-large-language-models/Recently, Meta announced the release of a new AI language generator called LLaMA. While tech enthusiasts have been primarily focused on language models developed by Microsoft, Google, and OpenAI, LLaMA is a research tool designed to help researchers advance their work in the...Tue, 07 Mar 2023 00:00:00 GMTResearchhttps://tullie.ai/blog/evaluating-recommendations-map-mmr-ndcg/https://tullie.ai/blog/evaluating-recommendations-map-mmr-ndcg/Imagine you’re shown two ordered feeds of product recommendations from separate algorithms. In the first one (A) you’re shown: Nike sneakers, Adidas shorts, and an Apple Watch. In the second one (B) you’re shown the order: Apple Watch, Adidas shorts, and Nike Sneakers. -- Which feed is more relevant to you?Wed, 01 Mar 2023 00:00:00 GMTEvalshttps://tullie.ai/blog/sounding-the-secrets-of-audiolm/https://tullie.ai/blog/sounding-the-secrets-of-audiolm/Dive into the inner workings of Google AI’s awesome language modeling approach to audio generation!Fri, 24 Feb 2023 00:00:00 GMTResearchhttps://tullie.ai/blog/do-llms-reason/https://tullie.ai/blog/do-llms-reason/In recent years, Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP), enabling significant advancements in language understanding, text generation, and more. With the help of memorization and compositionality capabilities, LLMs can...Tue, 21 Feb 2023 00:00:00 GMTResearchhttps://tullie.ai/blog/tiktok-recommendations-secret-sauce/https://tullie.ai/blog/tiktok-recommendations-secret-sauce/Dive into the inner workings of TikTok’s awesome real-time recommendation system and learn what makes it one of the best in the field!Fri, 17 Feb 2023 00:00:00 GMTIRhttps://tullie.ai/blog/evaluating-recommendations-precision-recall/https://tullie.ai/blog/evaluating-recommendations-precision-recall/Imagine you’re given three movie recommendations from separate algorithms. In the first one (A) you’re given: The Terminator, James Bond, and Star Wars. In the second (B) you’re given: Cars, Toy Story, and Iron Man -- Which recommendation is more relevant to you?Tue, 07 Feb 2023 00:00:00 GMTEvalshttps://tullie.ai/blog/whisper-robust-asr/https://tullie.ai/blog/whisper-robust-asr/Open-AI released Whisper, an open source speech recognition model with human level robustness and accuracy on English language. Trained on 680,000 hours of multilingual and multitask supervised data collected from the web.Tue, 24 Jan 2023 00:00:00 GMTResearchhttps://tullie.ai/blog/why-your-feeds-are-getting-worse-over-time/https://tullie.ai/blog/why-your-feeds-are-getting-worse-over-time/If you’ve ever browsed TikTok’s For You Page, Facebook Newsfeed, Instagram Reels, Youtube Shorts, or any “infinite” scrolling social feed you have been served personalized recommendations. These feeds use algorithms to find out what content will keep you browsing the feed....Mon, 24 Oct 2022 00:00:00 GMTIRhttps://tullie.ai/blog/your-browsing-behavior-is-being-modeled-as-a-language/https://tullie.ai/blog/your-browsing-behavior-is-being-modeled-as-a-language/What if I told you that your web browsing is being modeled as a language in most of the web pages you visit? Let me show you why and how.Wed, 28 Sep 2022 00:00:00 GMTIRhttps://tullie.ai/blog/day-3-of-recsys2022-our-favorite-5-papers-and-talks/https://tullie.ai/blog/day-3-of-recsys2022-our-favorite-5-papers-and-talks/The last day at RecSys 2022 started with a session on Sessions and Interaction, moved on to Models and Learning to finish with Large-Scale Recommendations. Here are our favorite 5 papers and talks of the day.Thu, 22 Sep 2022 00:00:00 GMTResearchhttps://tullie.ai/blog/day-2-of-recsys2022-our-favorite-5-papers-and-talks/https://tullie.ai/blog/day-2-of-recsys2022-our-favorite-5-papers-and-talks/It’s been another fantastic day at RecSys 2022. Following the Women in RecSys Breakfast, the day started with a keynote from Catherine D’Ignazio and then throughout the day had the following sessions: Fairness & Privacy, Diversity & Novely, and Models and Learning I. Here are our favorite 5 papers and talks.Tue, 20 Sep 2022 00:00:00 GMTResearchhttps://tullie.ai/blog/day-1-of-recsys2022-our-favorite-5-papers-and-talks/https://tullie.ai/blog/day-1-of-recsys2022-our-favorite-5-papers-and-talks/We just wrapped up a fantastic first day at Recsys2022 in Seattle. Here are our favorite 5 papers and talks of the day.Mon, 19 Sep 2022 00:00:00 GMTResearchhttps://tullie.ai/blog/explore-vs-exploit/https://tullie.ai/blog/explore-vs-exploit/Problems with bias in recommendation systems and what you can do about them.Tue, 30 Aug 2022 00:00:00 GMTIRhttps://tullie.ai/blog/yann-lecun-a-path-towards-autonomous-machine-intelligence/https://tullie.ai/blog/yann-lecun-a-path-towards-autonomous-machine-intelligence/How could machines learn as efficiently as humans and animals?Wed, 24 Aug 2022 00:00:00 GMTResearchhttps://tullie.ai/blog/information-retrieval-systems-the-precursors-of-recommender-systems/https://tullie.ai/blog/information-retrieval-systems-the-precursors-of-recommender-systems/Not sure about the differences between information retrieval systems and recommender systems? Don't worry, we got you covered.Mon, 08 Aug 2022 00:00:00 GMTIRhttps://tullie.ai/blog/data-centric-ai-for-ranking/https://tullie.ai/blog/data-centric-ai-for-ranking/Data quality and volume is what makes rankings algorithms at big-tech so seamless. How can you create the same experiences with the data you have? Data-centric AI may be the answer!Tue, 12 Jul 2022 00:00:00 GMTIRhttps://tullie.ai/blog/why-airbnb-made-such-a-big-deal-about-categories/https://tullie.ai/blog/why-airbnb-made-such-a-big-deal-about-categories/From Search based to Discovery firstThu, 19 May 2022 00:00:00 GMTIR