Scaling Feature Engineering Pipelines with Feast and Ray

challenge involving the construct of propensity fashions to foretell clients’ potential purchases, I encountered characteristic engineering points that I had seen quite a few instances earlier than.

These challenges could be broadly categorised into two classes:

1) Insufficient Characteristic Administration

• Definitions, lineage, and variations of options generated by the staff weren’t systematically tracked, thereby limiting characteristic reuse and reproducibility of mannequin runs.
• Characteristic logic was manually maintained throughout separate coaching and inference scripts, resulting in a threat of inconsistent options for coaching and inference (i.e., training-serving skew)
• Options have been saved as flat information (e.g., CSV), which lack schema enforcement and assist for low-latency or scalable entry.

2) Excessive Characteristic Engineering Latency

• Heavy characteristic engineering workloads typically come up when coping with time-series knowledge, the place a number of window-based transformations have to be computed.
• When these computations are executed sequentially quite than optimized for parallel execution, the latency of characteristic engineering can enhance considerably.

On this article, I clearly clarify the ideas and implementation of characteristic shops (Feast) and distributed compute frameworks (Ray) for characteristic engineering in manufacturing machine studying (ML) pipelines.

Contents

(1) Example Use Case
(2) Understanding Feast and Ray
(3) Roles of Feast and Ray in Feature Engineering
(4) Code Walkthrough

You will discover the accompanying GitHub repo here.

(i) Goal

As an example the capabilities of Feast and Ray, our instance situation entails constructing an ML pipeline to coach and serve a 30-day buyer buy propensity mannequin.

(ii) Dataset

We are going to use the UCI Online Retail dataset (CC BY 4.0), which contains buy transactions for a UK on-line retailer between December 2010 and December 2011.

(iii) Characteristic Engineering Strategy

We will maintain the characteristic engineering scope easy by limiting it to the next options (based mostly on a 90-day lookback window until in any other case acknowledged):

Recency, Frequency, Financial Worth (RFM) options

• recency_days: Days since final buy
• frequency: Variety of distinct orders
• financial: Whole financial spend
• tenure_days: Days since first-ever buy (all-time)

Buyer behavioral options

• avg_order_value: Imply spend per order
• avg_basket_size: Imply variety of objects per order
• n_unique_products: Product range
• return_rate: Share of cancelled orders
• avg_days_between_purchases: Imply days between purchases

(iv) Rolling Window Design

The options are computed from a 90-day window earlier than every cutoff date, and buy labels (1 = at the very least one buy, 0 = no buy) are computed from a 30-day window after every cutoff.

On condition that the cutoff dates are spaced 30 days aside, it produces 9 snapshots from the dataset:

(i) About Feast

Firstly, let’s perceive what a characteristic retailer is. 

A characteristic retailer is a centralized knowledge repository that manages, shops, and serves machine studying options, performing as a single supply of fact for each coaching and serving.

Characteristic shops provide key advantages in managing characteristic pipelines:

• Implement consistency between coaching and serving knowledge
• Forestall knowledge leakage by making certain options use solely knowledge out there on the time of prediction (i.e., point-in-time right knowledge)
• Enable cross-team reuse of options and have pipelines
• Observe characteristic variations, lineage, and metadata for governance

Feast (quick for Feature Store) is an open-source characteristic retailer that delivers characteristic knowledge at scale throughout coaching and inference.

It integrates with a number of database backends and ML frameworks that may work throughout or off cloud platforms.

Feast helps each on-line (for real-time inference) and offline (for batch predictions), although our focus is on offline options, as batch prediction is extra related for our buy propensity use case.

(ii) About Ray

Ray is an open-source general-purpose distributed computing framework designed to scale ML purposes from a single machine to massive clusters. It may well run on any machine, cluster, cloud supplier, or Kubernetes.

Ray presents a spread of capabilities, and the one we’ll use is the core distributed runtime referred to as Ray Core. 

Ray Core offers low-level primitives for the parallel execution of Python features as distributed duties and for managing duties throughout out there compute assets.

Let’s have a look at the areas the place Feast and Ray assist tackle characteristic engineering challenges.

(i) Characteristic Retailer Setup with Feast

For our case, we’ll arrange an offline characteristic retailer utilizing Feast. Our RFM and buyer habits options will likely be registered within the characteristic retailer for centralized entry.

In Feast terminology, offline options are additionally termed as ‘historic’ options

(ii) Characteristic Retrieval with Feast and Ray

With our Feast characteristic retailer prepared, we will allow the retrieval of related options from it throughout each phases of mannequin coaching and inference.

We should first be clear about these three ideas: Entity, Characteristic, and Characteristic View.

• An entity is the first key used to retrieve options. It mainly refers back to the identifier “object” for every characteristic row (e.g., user_id, account_id, and so forth)
• A characteristic is a single typed attribute related to every entity (e.g., avg_basket_size)
• A characteristic view defines a bunch of associated options for an entity, sourced from a dataset. Consider it as a desk with a major key (e.g., user_id) being coupled with related characteristic columns.

Occasion timestamps are an integral part of characteristic views because it permits usto generate point-in-time right characteristic knowledge for coaching and inference.

Say we now wish to get hold of these offline options for coaching or inference. Right here’s how it’s carried out:

1. An entity DataFrame is first created, containing the entity keys and an occasion timestamp for every row. It corresponds to the 2 left-most columns in Fig. 5 above.
2. A point-in-time right be a part of happens between the entity DataFrame and the characteristic tables outlined by the completely different Characteristic Views

The output is a mixed dataset containing all of the requested options for the desired set of entities and timestamps.

So the place does Ray are available right here?

The Ray Offline Store is a distributed compute engine that permits quicker, extra scalable characteristic retrieval, particularly for big datasets. It does so by parallelizing knowledge entry and be a part of operations:

• Information (I/O) Entry: Distributed knowledge reads by splitting Parquet information throughout a number of employees, the place every employee reads a unique partition in parallel
• Be part of Operations: Splits the entity DataFrame so that every partition independently performs temporal joins to retrieve the characteristic values per entity earlier than a given timestamp. With a number of characteristic views, Ray parallelizes the computationally intensive joins to scale effectively.

(iii) Characteristic Engineering with Ray

The characteristic engineering operate for producing RFM and buyer habits options have to be utilized to every 90-day window (i.e., 9 impartial cutoff dates, every requiring the identical computation).

Ray Core turns every operate name right into a remote task, enabling the characteristic engineering to run in parallel throughout out there cores (or machines in a cluster). 

(4.1) Preliminary Setup

We set up the next Python dependencies:

feast[ray]==0.60.0
openpyxl==3.1.5
psycopg2-binary==2.9.11
ray==2.54.0
scikit-learn==1.8.0
xgboost==3.2.0

As we’ll use PostgreSQL for the characteristic registry, ensure that Docker is put in and working earlier than working docker compose up -d to start out the PostgreSQL container.

(4.2) Put together Information 

Apart from data ingestion and cleaning, there are two preparation steps to execute:

• Rolling Cutoff Generation: Creates 9 snapshots spaced 30 days aside. Every cutoff date defines a coaching/prediction level at which options are computed from the 90 days previous it, and goal labels are computed from the 30 days after it.
• Label Creation: For every cutoff, create a binary goal label indicating whether or not a buyer made at the very least one buy throughout the 30-day window after the cutoff.

(4.3) Run Ray-Based mostly Characteristic Engineering

After defining the code to generate RFM and customer behavior options, let’s parallelize the execution utilizing Ray for every rolling window.

We begin by making a operate (compute_features_for_cutoff) to wrap all of the related characteristic engineering steps for each cutoff:

The @ray.distant decorator registers the operate as a distant activity to be run asynchronously in separate employees.

The info preparation and have engineering pipeline is then run as follows:

Right here’s how Ray is concerned within the pipeline:

• ray.init() initiates a Ray cluster and permits distributed execution throughout all native cores by default. 
• ray.put(df) shops the cleaned DataFrame in Ray’s shared reminiscence (aka distributed object retailer) and returns a reference (ObjectRef) so that each one parallel duties can entry the DataFrame with out copying it. This helps to enhance reminiscence effectivity and activity launch efficiency
• compute_features_for_cutoff.distant(...) sends our characteristic computation duties to Ray’s scheduler, the place Ray assigns every activity to a employee for parallel execution and returns a reference to every activity’s output.
• futures = [...] shops all references returned by every .distant() name. They characterize all of the in-flight parallel duties which were launched
• ray.get(futures) retrieves all of the precise return values from the parallel activity executions at one go
• The script then extracts and concatenates per-cutoff RFM and habits options into two DataFrames, saves them as Parquet information domestically
• ray.shutdown() releases the assets allotted by stopping the Ray runtime

Whereas our options are saved domestically on this case, do observe that offline characteristic knowledge is often saved in knowledge warehouses or knowledge lakes (e.g., S3, BigQuery, and so forth) in manufacturing settings.

(4.4) Arrange Feast Characteristic Registry

Thus far, we’ve got coated the transformation and storage features of characteristic engineering. Allow us to transfer on to the Feast characteristic registry.

A characteristic registry is the centralized catalog of characteristic definitions and metadata that serves as a single supply of fact for characteristic info.

There are two key parts within the registry setup: Definitions and Configuration.

Definitions

We first outline the Python objects to characterize the options engineered to date. For instance, one of many first objects to find out is the Entity (i.e., the first key that hyperlinks the characteristic rows):

Subsequent, we outline the knowledge sources wherein our characteristic knowledge are saved:

Notice that the timestamp_field is vital because it permits right point-in-time knowledge views and joins when options are retrieved for coaching or inference.

After defining entities and knowledge sources, we will outline the characteristic views. On condition that we’ve got two units of options (RFM and buyer habits), we anticipate to have two characteristic views:

The schema (area names, dtypes) is necessary for making certain that characteristic knowledge is correctly validated and registered.

Configuration

The characteristic registry configuration is outlined in a YAML file referred to as feature_store.yaml:

The configuration tells Feast what infrastructure to make use of and the place its metadata and have knowledge reside, and it usually contains the next:

• Mission identify: Namespace for challenge
• Supplier: Execution setting (e.g., native, Kubernetes, cloud)
• Registry location: Location of characteristic metadata storage (file or databases like PostgreSQL)
• Offline retailer: Location from which historic options knowledge is learn
• On-line retailer: Location from which low-latency options are served (not related in our case)

In our case, we use PostgreSQL (working in a Docker container) for the characteristic registry and the Ray offline retailer for optimized characteristic retrieval.

We use PostgreSQL as an alternative of native SQLite to simulate production-grade infrastructure for the characteristic registry setup, the place a number of providers can entry the registry concurrently

Feast Apply

As soon as definitions and configuration are arrange, we run feast apply to register and synchronize the definitions with the registry and provision the required infrastructure.

The command could be discovered within the Makefile:

# Step 2: Register Feast characteristic definitions in PostgreSQL registry
apply:
 cd feature_store && feast apply

(4.5) Retrieve Options for Mannequin Coaching

As soon as our characteristic retailer is prepared, we proceed with coaching the ML mannequin. 

We begin by creating the entity backbone for retrieval (i.e., the 2 columns of customer_id and event_timestamp), which Feast makes use of to retrieve the right characteristic snapshot.

We then execute the retrieval of options for mannequin coaching at runtime:

• FeatureStore is the Feast object that’s used to outline, create, and retrieve options at runtime
• get_historical_features() is designed for offline characteristic retrieval (versus get_online_features()), and it expects the entity DataFrame and the checklist of options to retrieve. The distributed reads and point-in-time joins of characteristic knowledge happen right here.

(4.7) Retrieve Options for Inference

We finish off by producing predictions from our educated mannequin.

The characteristic retrieval codes for inference are largely just like these for coaching, since we’re reaping the advantages of a constant characteristic retailer.

The primary distinction comes from the completely different cutoff dates used.

Wrapping It Up

Characteristic engineering is an important part of constructing ML fashions, but it surely additionally introduces knowledge administration challenges if not correctly dealt with.

On this article, we clearly demonstrated tips on how to use Feast and Ray to enhance the administration, reusability, and effectivity of characteristic engineering.

Understanding and making use of these ideas will allow groups to construct environment friendly ML pipelines with scalable characteristic engineering capabilities.