Riskified Improves Agility and DevOps Maturity with Faros AI

Date: July 23, 2024  |  Reading Time: 12 min

Summary: Discover how Riskified, a leader in ecommerce fraud and risk intelligence, leveraged Faros AI to strengthen engineering autonomy, accountability, and business results through a data-driven approach.

Riskified engineering team

Key Outcomes at a Glance

Chapters

  1. Strengthening Engineering Culture at Riskified
  2. A Commitment to Business Value and Relentless Improvement
  3. The Goal: Improve Processes and Reduce Risk
  4. Becoming Data-Driven in Retros and Planning
  5. Improving Quality of Delivery to Protect Profitability
  6. Transitioning to Data-Driven Tech QBRs
  7. Riskified’s Advice for Implementing Engineering Metrics

Strengthening Engineering Culture at Riskified

Riskified is an enterprise-grade fraud and risk intelligence ecommerce solution that leverages big data and machine learning to approve transactions and combat fraud. During a period of rapid growth, the engineering team doubled in size, challenging leadership to maintain speed and quality while scaling.

To support a culture of Lean, Agile, and DevOps, Riskified selected Faros AI to provide the visibility needed for highly autonomous teams to optimize efficiency and effectiveness.

A Commitment to Business Value and Relentless Improvement

“At Riskified, we have a strong DevOps culture, characterized by a mindset of end-to-end ownership. Our teams are responsible for their uptime, quality, performance, and even cost.”
— Shai Peretz, SVP of Engineering, Riskified

Shai Peretz, SVP of Engineering, emphasizes that maintaining this culture is challenging during rapid growth. Faros AI supplements intuition with scientific, objective data, adding a new layer of visibility and supporting the tenet of relentless improvement. Faros AI also helps decentralize decision-making, empowering teams to act as subject matter experts.

The Goal: Improve Processes and Reduce Risk

Tal Levinger, Agile Program Manager, led the selection and implementation of Faros AI. The goal: establish a baseline, assess strengths and weaknesses, and enable teams to independently track and improve metrics.

Becoming Data-Driven in Retros and Planning

Engineering team leader Elad Kochavi pioneered the use of Faros AI dashboards in sprint retrospectives. Teams now:

“Our transition to data-driven retros has energized and motivated the team; they love seeing the impact of their efforts in the charts.”
— Elad Kochavi, Engineering Team Leader, Riskified

Improving Quality of Delivery to Protect Profitability

Reliability and quality are critical for Riskified, as some offerings assume financial responsibility for fraud. Faros AI enables:

“The metrics in Faros AI have more complexity, depth, and dimensions because they draw directly from the code itself—which is always more reliable and preferred.”
— Shai Peretz

Transitioning to Data-Driven Tech QBRs

Faros AI dashboards are now used in Tech QBRs, mapping engineering work to business value and supporting bottom-up improvement. This data-driven approach:

“Having the insights from Faros AI has increased my confidence when working with leadership on priorities, capacity, and resources.”
— Elad Kochavi

Riskified’s Advice for Implementing Engineering Metrics

Related Customer Stories

See What Faros AI Can Do for You

Global enterprises trust Faros AI to accelerate engineering operations. Request a demo to see how you can achieve similar results.

Customers
July 23, 2024
12
min read

Riskified Improves Agility and DevOps Maturity with a Data-Driven Approach Powered by Faros AI

Discover how Riskified’s engineering organization strengthens team autonomy and accountability to achieve outstanding business results in the competitive cybersecurity market.

Naomi Lurie
Naomi Lurie
Banner image of Shai Peretz, SVP Engineering at Riskified on a dark blue background with the blog title: Riskified improves agility and devops maturity with a data-driven approach, powered by Faros AI.
12
min read
Browse Chapters
12
min read
Riskified Improves Agility and DevOps Maturity with a Data-Driven Approach Powered by Faros AI
Strengthening engineering culture at Riskified, a leader in ecommerce fraud and risk intelligence
A commitment to business value is a commitment to relentless improvement
The goal: To improve processes and reduce risk
Becoming data-driven in retros and planning
Improving quality of delivery to protect profitability
Transitioning to data-driven tech QBRs
Riskified’s advice for implementing engineering metrics
Share
Twitter
Linkedin
Mail
Copy Link
https://www.faros.ai/blog/customer-stories-riskified
Share
Twitter
Linkedin
Mail
Copy Link
https://www.faros.ai/blog/customer-stories-riskified
July 23, 2024

Riskified Improves Agility and DevOps Maturity with a Data-Driven Approach Powered by Faros AI

  • A new layer of visibility from Faros AI helps Riskified measure itself and continuously improve as it scales.
  • Customizable dashboards are used in retros and planning sessions, to help increase predictability and reliability, team by team, sprint by sprint.
  • Decision-making has been confidently decentralized to the team level, where domain leaders have a clear understanding of priorities, capacity, and resources.
  • Tech QBR meetings have become data-driven, helping every engineering team map its contributions to business results.
Stylized quote on a white background:

Strengthening engineering culture at Riskified, a leader in ecommerce fraud and risk intelligence

Riskified is an enterprise-grade fraud and risk intelligence ecommerce solution that efficiently combats fraud and curbs policy abuse. It boosts merchant revenue by leveraging big data and machine learning to approve transactions that merchants might otherwise decline.

At a time of accelerated growth, the engineering organization doubled in size. The challenge facing its leadership was maintaining the commitment to delivering business value with speed and quality as they scaled. To that end, they sought a solution that would help strengthen the culture of Lean, Agile, and DevOps across the growing team.

Riskified chose Faros AI to provide the visibility that empowers its highly autonomous teams to optimize their efficiency and effectiveness.

A commitment to business value is a commitment to relentless improvement

Shai Peretz, SVP of Engineering at Riskified, joined the company in 2021. A Lean, Agile, and DevOps enthusiast and mentor, Shai was impressed with Riskified’s existing engineering culture.

“At Riskified, we have a strong DevOps culture, characterized by a mindset of end-to-end ownership. Our teams are responsible for their uptime, quality, performance, and even cost,” describes Shai.

But from his years of experience, Shai also knew that maintaining the culture is hard when you’re growing and hiring extremely fast. “That’s when you need tools to help supplement gut feelings with more scientific, objective data,” says Shai. “Faros AI adds a new layer of visibility that helps the organization measure itself and continuously improve. It reflects our tenet of relentless improvement.”

Shai strongly believes that decisions should be made by subject matter experts and delegated to the teams. He jokes that if a decision is escalated to him, he conducts a retrospective to understand what went wrong. He never wants to become a bottleneck, and — to that end — he invests in tools like Faros AI to help decentralize decision-making.

The goal: To improve processes and reduce risk

Tal Levinger is Riskified’s Agile Program Manager, who reports to Shai. Her role is to implement a Lean-Agile culture, help teams thrive, and implement tools and processes that improve their workflows and outcomes. It was natural for her to lead the selection and implementation of a visibility solution.

“Our goal was to first determine the baseline of the organization, get the numbers. Then we would assess the good and the bad, set some action items based on the metrics, and allow each team to track them and improve them independently,” says Tal. “For example, if we didn’t deliver well, we wanted to understand why — was it due to resource constraints, poor definition, or something else?”

Tal evaluated many tools, but Riskified chose Faros AI for three reasons.

“First, Faros AI was the only tool that met Riskified’s stringent security and compliance requirements. It was the only solution that could accommodate our data ingestion preferences and allow us to run the connectors ourselves to avoid providing system credentials to a third party,” says Tal.

Stylized quote on a white background:

“Second, Faros AI was the only software engineering intelligence platform that had the customizability and flexibility to meet us where we are and produce the metrics based on how we work today. All our tools are highly customized and we needed a vendor with the expertise to extract and standardize the data for reporting. For example, the Faros AI team provided a custom connector to ingest events from our bespoke deployment pipeline, so we could link PRs and commits to deployments for a complete picture.

“Third, we wanted to be able to easily build our own metrics and dashboards, beyond what comes out of the box with most vendors,” continues Tal. “Faros AI gives us the flexibility to create new metrics and views as needed.”

Tal also appreciated that Faros AI let Riskified select the granularity of the metrics. “We use Faros AI to learn and improve team by team. I like that we’re able to configure our dashboards to look at what the teams are doing, not the individuals.”

Becoming data-driven in retros and planning

Elad Kochavi is an engineering team leader at Riskified. He was one of the earliest domain leaders to run his sprint retrospectives off of Faros AI dashboards. Elad’s goal is for the team to learn and improve together.

In their retrospectives, Elad's team examines their metrics to ensure they’re focusing on the right priorities, becoming better at estimation, and keeping WIP under control. “Running a retro with concrete data naturally opens the door to more creative and critical thinking,” says Elad.

They also measure KTLO (keeping the lights on) work vs. value-adding feature work, something that was not possible before Faros AI.

“The biggest benefit we see is that we no longer rely on gut feelings to set our action items,” says Elad. “We now have a combined picture from all the tools we use and can do much more sophisticated analysis in place of the naive and simplified views in Jira. Our transition to data-driven retros has energized and motivated the team; they love seeing the impact of their efforts in the charts.”

Like Tal, Elad loves the flexibility he has to build his own metrics. “The beauty of Faros AI is that we can keep adding more and more metrics if something comes out of our retros that we want to analyze or improve. It’s pretty simple and intuitive for us to pull together the data from our different sources, build new charts, and add them to our retro dashboard.”

Stylized quote on a white background. “The beauty of Faros AI is that we can keep adding more and more metrics if something comes out of our retros that we want to analyze or improve.

Improving quality of delivery to protect profitability

Reliability and quality are critical to Riskified, as some of the company’s offerings assume the financial responsibility for ecommerce fraud. A bug in production can have significant financial repercussions.

Thus, Tal focuses on how to improve Riskified processes to reduce risk, minimize dependencies, and streamline handoffs. Domain leaders like Elad focus on improving deployment quality, to minimize the number of times a rollback or hotfix is required.

For that reason, Riskified leverages a wide set of metrics on Faros AI, covering DORA Metrics, engineering productivity, agile health, software quality metrics like test coverage, and lagging indicators like production incidents.

“The ability to connect workflows from all our different data sources from development through production is so significant in understanding our performance. Jira only tells you how a task’s status has changed over time," says Shai.

"The metrics in Faros AI have more complexity, depth, and dimensions because they draw directly from the code itself—which is always more reliable and preferred. For example, with Faros AI I can understand how many times we deployed to test environments or how many times a task was reopened due to unclear requirements.”

Transitioning to data-driven tech QBRs

Shai introduced Faros AI at Riskified for the teams to have a mirror for themselves, to drive bottom-up improvement. However, the data is proving valuable to the ELT as well. Today, Faros AI dashboards are used in Tech QBRs.

“In our QBRs, metrics in Faros AI help map engineering’s work to business value. The excellence with which our engineering teams deliver and operate can be tied directly to the lagging results, like helping the business acquire, retain, upsell or increase customer satisfaction,” says Shai.

Elad also feels more empowered in these discussions with leadership. “Having the insights from Faros AI has increased my confidence when working with leadership on priorities, capacity, and resources,” he says.

Shai is pleased with how the data-driven mindset is strengthening the engineering function. “When an organization becomes data-driven, they’re more dialed into why they are doing these things and developing these specific features. That helps them learn the language of the business, foster stronger cross-functional collaboration, and become better leaders.”

Riskified’s advice for implementing engineering metrics

Having gone through the implementation and adoption of a software engineering intelligence platform, Shai shares his advice with others embarking on a similar journey.

First, generating visibility is very dependent on the tools you use and how you use them. This is where a lot of human expertise is required, to decide how to best reflect your business processes and workflows in your metrics. “No tool can magically do this on your behalf or create data where none exists, so be prepared to devote some time and thought to these questions,” counsels Shai.

Second, get creative when it comes to motivating teams to improve. For example, the Riskified office has monitors in the kitchenettes and around the office that display company info and announcements. Tal had the idea to display some engineering metrics from Faros AI on them, to show how the tech org was meeting business goals. “I’ve gone into the kitchen a few times and found people discussing the stats. It motivates them to dive deeper and think about improvements we can make, which is fantastic,” says Shai.

Finally, the entrance of AI-driven development tools like GitHub Copilot demands better visibility into tech productivity and cost. “While tools like GitHub Copilot have the potential to increase productivity, evaluating their impact scientifically will help build the business case for the investment.”

Back to Blog Posts
AI Is Everywhere. Impact Isn’t.
75% of engineers use AI tools—yet most organizations see no measurable performance gains.

Read the report to uncover what’s holding teams back—and how to fix it fast.
Read the report
AI Productivity Paradox Report 2025
Discover the Engineering Productivity Handbook
How to build a high-impact program that drives real results.
‍
What to measure and why it matters.

And the 5 critical practices that turn data into impact.
Get the handbook
The cover of The Engineering Productivity Handbook on a turquoise background
Naomi Lurie

Naomi Lurie

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

Connect
closed
open
Want to learn more about Faros AI?

Fill out this form and an expert will reach out to schedule time to talk.

Loading calendar...
An illustration of a lighthouse in the sea

Thank you!

A Faros AI expert will reach out to schedule a time to talk.
P.S. If you don't see it within one business day, please check your spam folder.
Oops! Something went wrong while submitting the form.

More articles for you

Editor's Pick
Customers
DevProd
6
MIN READ

Vimeo Relies on Faros AI for Efficient and Predictable Software Delivery

Learn how Vimeo’s engineering organization improved lead times, delivery metrics, and GenAI adoption with centralized visibility and insights into SDLC workflows.
December 20, 2024
Editor's Pick
DevProd
Customers
10
MIN READ

Why Autodesk Chose a Platform Approach to Developer Productivity and GenAI Impact

Autodesk shares its key learnings from building an internal developer platform with an integrated visibility plane to optimize the software development lifecycle.
September 6, 2024
Editor's Pick
Customers
10
MIN READ

SmartBear’s Software Engineering Scales to Support Rapid Growth by Measuring Outcomes with Faros AI

Learn how SmartBear monitors portfolio investments to drive growth, improve predictability, and boost customer satisfaction
November 1, 2023

See what Faros AI can do for you!

Global enterprises trust Faros AI to accelerate their engineering operations. Give us 30 minutes of your time and see it for yourself.

Request a demo