Introduction
Snap is committed to bringing a world class Augmented Reality (AR) and ecommerce experience to our advertisers and users. At the Snap Partners Summit in April 2022, we saw users trying on different products via Snap lenses and directly shopping in-app. In this post, we will walk through how the Catalog Platform Team at Snap brings AR assets onto our platform in order to make this experience possible.
Background/Motivation
To advertise with a set of products on Snap, advertisers aggregate eligible product metadata fields such as description, title, price, etc. into a single catalog. To advertise with AR eligible products, advertisers also need to provide links to AR asset files in addition to the required product data for each product in the catalog. Historically, a single data source known as a data feed contained all of this product data.
However, some advertisers who wish to advertise with AR eligible products, such as using product lenses, don’t have the capacity to generate AR asset files for their products. For these advertisers, they often partner with third party AR/3D asset providers to create the appropriate asset files. Since it is not always feasible for the advertisers to edit their existing data feed to include links to the generated asset files, our feed system must now support the construction of a product catalog from two data feeds: one containing product metadata (advertiser catalog) and one only containing the AR assets.
In this new world, the product feed that contains at least the set of required product metadata fields is known as a primary feed. The feed that supplements the primary feed is known as the supplemental feed. The only requirement for a supplemental feed is that it must contain product IDs referenced in the primary feed. By definition, a supplemental feed may only add to a primary feed’s data, as it does not make sense to process a supplemental feed without the context of a primary feed. Lastly, a primary feed’s values always take precedence over those of a supplemental feed’s.
Let’s take a look at the two main components we made changes to support supplemental feed: 1. feed processing system 2. the core catalog service.
System Overview
Our feed ingestion system is built on top of Temporal, an open source project “which helps with service/workflow orchestration”. Our feed ingestion Temporal Workflow consists of the following steps: downloading the feed file, ingesting and partitioning the feed file into segments, and submitting the product data in each partition via a gRPC call to our catalog service. The catalog service then accepts the products and facilitates them through the product ingestion pipeline and persists them into our database. Lastly, our vending service surfaces all eligible products to downstream services.
In essence, the steps to ingest a supplemental feed do not differ much from that of primary feed. However, we now add a new step (via a Temporal Activity) to the Temporal workflow to ensure that there exists at least one successful primary feed upload.
Then comes the next challenge: How do we construct a full product entity from two data sources? Recall that the feed ingestion system uses gRPC to communicate with the core catalog system. Google Protobuf provides a concept of FieldMask, which represents a set of symbolic field paths in a Google Protobuf message (See Netflix’s FieldMask blog for an excellent explanation). Think of a FieldMask as a signature of all the fields within a data feed. For example, if a feed contains only the commerce required fields, then the FieldMask for that feed would look like the code snippet below.
Algorithmic Deep Dive
Knowing the FieldMask representation and the feed type (primary or supplemental) of an incoming feed, the catalog service can construct fully fledged products for the catalog.
In product creation, we persist the new product metadata along with a mapping of the primary feed ID to its FieldMask, as the primary feed is the only feed type permitted to create a new product (see example below). If the advertiser maintains only a primary feed for the catalog, then any subsequent update is just a matter of updating the product data and the FieldMask for the primary feed. The only time this mapping changes is when a new field is added to or removed from the primary feed.
When an advertiser triggers a product update with a supplemental feed, we first retrieve the product’s primary feed ID and its primary feed FieldMask. For every field in the primary feed’s FieldMask, we preserve its existing value because the primary feed’s values cannot be overwritten by a supplemental feed. After which we finish constructing the next version of the product by populating the value for each field in the supplemental feed’s FieldMask. Once we update the product metadata and FieldMask mapping, we persist this information to the database and await the next feed run (see example below).
If at any time the supplemental feed attempts to overwrite a value previously provided by the primary feed, we ignore the value provided by the supplemental feed. Conversely, if the primary feed overwrites any value previously provided by the supplemental feed, we favor the primary feed’s value. At this time, we also update the value to that of the primary feed’s and update the primary feed’s FieldMask to indicate that the primary feed last populated the field.
Summary
With supplemental feeds, advertisers can build a catalog of products from multiple product data feeds, such as combining a third party generated AR asset feed with an advertiser product metadata feed to create AR enabled Snap ads. This project was an enormous undertaking where we retrofitted our existing feed system to make use of Protobuf FieldMask. Supplemental feeds launched in early 2022 and have already been adopted by many of our advertisers. If you’re an advertiser and want to start using supplemental feeds, please check out our commerce documentation here.