Model-Based Testing for a Secure Shopify Theme with XState - Djang0 - Developer

blog post thumbnail

SHARE

Reading Time: 7 minutes

The repository is accessible and has been updated with the latest major XState version.

Model Based Testing applied to a Shopify Liquid Theme
https://github.com/djang0dev/mbt-shopify-liquid-theme
0 forks.
1 stars.
0 open issues.

Recent commits:

first commit, Kinane

Table of Contents

Model-based testing (MBT) involves creating a model that describes how a system should work and then using this model to generate tests automatically. The model is often a state machine, representing the stages or states the system can reach and the transitions between them. Ok this a bit boring, A concrete example would be to represent the different possible paths of a user on an e-commerce site (a bit like when you spy on Hotjar and see the navigation steps of a customer).

Basic Example

Simple representation of a basic machine which goes to home page and then shop all page

Those charts are representations from Stately; it's generated based on my machine's code, there's also a vscode plugin to represent your machine

From this model, we can derive the states and transitions:

States

onHomePage: User will be redirected to the Home Page
inShopAllPage: The user is on the “Shop All” page.

Events

HEADER_MENU_GO_TO_HOME_PAGE: Transition from any page back to the home page via the header menu.
HERO_GO_TO_SHOP_ALL: Transition from the home page to the “Shop All” page via the hero button or section.

With these events and states, we can impute the playwrights' related actions.

{
  states: {
    onHomePage: async () => {
      await page.goto(`${baseShopUrl}/`);
    },
    inShopAllPage: async () => {
      await page.waitForURL(`${baseShopUrl}/collections/all`);
    },
  },
  events: {
    HERO_GO_TO_SHOP_ALL: async () => {
      const heroButton = await page.$(".banner a");
      if (!heroButton) throw new Error("Hero button not found");
      await heroButton.click();
    },
    HEADER_MENU_GO_TO_HOME_PAGE: async () => {
      const homeButton = await page.$("#HeaderMenu-home");
      if (!homeButton) throw new Error("Home button not found");
      await homeButton.click();
    },
  },
}

How many paths (and thus tests) do you think this will generate? 1? 2? even more?

Answer: Only one 😅

In MBT, each state is unique, so the return path is not set up as a standard

To do so, we need to create a “unique” state:

CleanShot 2023 10 18 at 11.40.25@2x — Model-Based Testing for a Secure Shopify Theme with XState 5

Here we go, using @xstate/test and playwright

import { test } from "@playwright/test"
import { createTestMachine, createTestModel } from "@xstate/test"
import { e2eConfig } from "./config"

const machine = createTestMachine({
  id: "basicHomePage",
  initial: "onHomePage",
  states: {
    onHomePage: {
      on: {
        HERO_GO_TO_SHOP_ALL: {
          target: "#basicHomePage.inShopAllPage",
          reenter: false,
        },
      },
    },
    inShopAllPage: {
      on: {
        HEADER_MENU_GO_TO_HOME_PAGE: {
          target: "#basicHomePage.inHomePageDestination",
          reenter: false,
        },
      },
    },
    inHomePageDestination: {
      type: "final",
    },
  },
})

createTestModel(machine)
  .getSimplePaths()
  .forEach((path) => {
    test(path.description, async ({ page }) => {
      await path.test({
        states: {
          onHomePage: async () => {
            await page.goto(`${e2eConfig.localBaseShopUrl}/`)
          },
          inHomePageDestination: async () => {
            await page.waitForURL(`${e2eConfig.localBaseShopUrl}/`)
          },
          inShopAllPage: async () => {
            await page.waitForURL(`${e2eConfig.localBaseShopUrl}/collections/all`)
          },
          inShopAllPageDestination: async () => {
            await page.waitForURL(`${e2eConfig.localBaseShopUrl}/`)
          },
        },
        events: {
          HERO_GO_TO_SHOP_ALL: async () => {
            const heroButton = await page.$(".banner a")
            if (!heroButton) throw new Error("Hero button not found")
            await heroButton.click()
          },
          HEADER_MENU_GO_TO_HOME_PAGE: async () => {
            const homeButton = await page.$("#HeaderMenu-home")
            if (!homeButton) throw new Error("Home button not found")
            await homeButton.click()
          },
        },
      })
    })
  })

In this way, We go back to the first theoretical state onHomePage:

The user starts on the home page (onHomePage).
The user clicks on the hero button or section (HERO_GO_TO_SHOP_ALL).
The user is navigated to the “Shop All” page (inShopAllPage).
The user then clicks on the header menu to return to the home page (HEADER_MENU_GO_TO_HOME_PAGE).
The user ends back on the home page (inHomePageDestination).

onHomePage -> HERO_GO_TO_SHOP_ALL -> inShopAllPage -> HEADER_MENU_GO_TO_HOME_PAGE -> inHomePageDestination

Round-trip result 🏎️

Declarative > Imperative

By specifying the various states and events, we give control over the generation of test steps to the model, rather than making it explicit ourselves.

Advanced Structure using Model-Based Testing on a Cart Drawer

We can now generate the user workflow that wants, for instance, to buy a product using different paths

Model-Based testing stately representation on a shopify Cart Drawer — Model-Based Testing for a Secure Shopify Theme with XState 6

import { expect, test } from "@playwright/test"
import { createTestModel } from "@xstate/test"
import { setup } from "xstate"
import { e2eConfig } from "./config"
import { genPwMetas, generateXstateStateAndEventsFromPw } from "./utils"

const singleProductPageBaseUrl = `${e2eConfig.localBaseShopUrl}/products/the-3p-fulfilled-snowboard`

const advancedAddToCartMachine = setup({
  types: {
    events: {} as
      | {
          type: "ADD_TO_CART"
        }
      | {
          type: "BUY_NOW"
        }
      | {
          type: "SHOW_CART"
        }
      | {
          type: "CLICK_ON_VIEW_CART_PAGE_BUTTON"
        }
      | {
          type: "CLICK_ON_HIDE_CART"
        }
      | {
          type: "CLICK_ON_CART_ICON"
        }
      | {
          type: "CHECK_PRODUCT_ADDED_TO_CART"
        }
      | {
          type: "GO_TO_CHECKOUT"
        },
  },
}).createMachine({
  initial: "onProductPage",
  id: "atcMachine",
  context: {},
  states: {
    // This act as our entrypoint
    onProductPage: {
      on: {
        ADD_TO_CART: "cartDrawer.visible",
        BUY_NOW: "inCheckout",
      },
    },
    cartDrawer: {
      initial: "hidden",
      states: {
        hidden: {
          on: {
            CLICK_ON_CART_ICON: "#atcMachine.inCartPage",
          },
        },
        visible: {
          on: {
            CLICK_ON_HIDE_CART: "hidden",
            CLICK_ON_CART_ICON: "#atcMachine.inCartPage",
            CLICK_ON_VIEW_CART_PAGE_BUTTON: "#atcMachine.inCartPage",
          },
        },
      },
    },
    productAddedToCart: {
      on: {
        GO_TO_CHECKOUT: "inCheckout",
      },
    },
    inCartPage: {
      on: {
        GO_TO_CHECKOUT: "inCheckout",
      },
    },
    inCheckout: {
      type: "final",
    },
  },
})

// Not quite sure of this pattern compared to simply use `meta`
const gen = genPwMetas<typeof advancedAddToCartMachine>((page) => {
  return {
    states: {
      inCartPage: {
        meta: {
          urlRegex: new RegExp(
            `^${e2eConfig.localBaseShopUrl.replace(/[-\/\^$*+?.()|[\]{}]/g, "\\$&")}/cart`,
            "i",
          ),
          waitFor: true,
          expectUrl: true,
        },
      },
      onProductPage: {
        decoratedHandler: async () => {
          await page.goto(singleProductPageBaseUrl)
          await page.waitForURL(singleProductPageBaseUrl)
        },
      },
      "cartDrawer.visible": async () => {
        await page.waitForSelector("#cart-notification")
        const cartDrawer = await page.$("#cart-notification")
        if (!cartDrawer) throw new Error("Cart drawer not found")
        const isActive = cartDrawer.evaluate((node) => node.classList.contains("active"))
        expect(isActive).toBeTruthy()
      },
      addedToCart: async () => {
        const cartItem = await page.$("#cart-notification-product")
        if (!cartItem) throw new Error("Cart item not found")
        const titleText = await cartItem.evaluate(async (node) => {
          const title = node.querySelector("h3")
          if (!title) throw new Error("Cart item title not found")
          return title.textContent
        })
        expect(titleText).toBe("The 3P Fulfilled Snowboard")
      },
      inCheckout: {
        meta: {
          urlRegex: new RegExp(
            `^(?:${e2eConfig.localBaseShopUrl.replace(
              /[-\/\^$*+?.()|[\]{}]/g,
              "\\$&",
            )}|${e2eConfig.webBaseShopUrl.replace(
              /[-\/\^$*+?.()|[\]{}]/g,
              "\\$&",
            )})/checkouts/.*$`,
            "i",
          ),
          waitFor: true,
          expectUrl: true,
        },
      },
      hidden: {},
    },
    events: {
      ADD_TO_CART: async () => {
        const addToCartButton = await page.$('button[name="add"]')
        if (!addToCartButton) throw new Error("Add to cart button not found")
        await addToCartButton.click()
      },
      GO_TO_CHECKOUT: async () => {
        const goToCheckoutButton = await page.$("#checkout")
        if (!goToCheckoutButton) throw new Error("Go to checkout button not found")
        await goToCheckoutButton.click()
      },
      BUY_NOW: async () => {
        const buyNowButton = await page.$(
          ".shopify-payment-button__button.shopify-payment-button__button--unbranded",
        )
        if (!buyNowButton) throw new Error("Buy now button not found")
        await buyNowButton.click()
      },
      SHOW_CART: async () => {
        const cartButton = await page.$("#cart-notification-button")
        if (!cartButton) throw new Error("Cart button not found")
        await cartButton.click()
      },
      CLICK_ON_VIEW_CART_PAGE_BUTTON: async () => {
        const viewCartPageButton = await page.$("#cart-notification-button")
        if (!viewCartPageButton) throw new Error("View cart page button not found")
        await viewCartPageButton.click()
      },
      CLICK_ON_HIDE_CART: async () => {
        const hideCartButton = await page.$(".cart-notification__close")
        if (!hideCartButton) throw new Error("Hide cart button not found")
        await hideCartButton.click()
      },
      CLICK_ON_CART_ICON: async () => {
        const cartIcon = await page.$("#cart-icon-bubble")
        if (!cartIcon) throw new Error("Cart icon not found")
        await cartIcon.click()
      },
    },
  }
})

createTestModel(advancedAddToCartMachine)
  .getSimplePaths()
  .forEach((path) => {
    test(path.description, async ({ page }) => {
      await path.test(generateXstateStateAndEventsFromPw(gen(page), page))
    })
  })

And generate a tiny factory for the related events and states to bind the playwright's logic:

const advancedAddToCartMachinePlaywrightStatesAndEventsFactory = (
  page: Page
): StateAndEvents => {
  return {
    states: {
      inCartPage: {
        decoratedHandler: async () => {},
        meta: {
          urlRegex: new RegExp(
            `^${baseShopUrl.replace(/[\/\^$*+?.()|[\]{}-]/g, "\\$&")}/cart`,
            "i"
          ),
          waitFor: true,
          expectUrl: true,
        },
      },
      onProductPage: {
        decoratedHandler: async () => {
          await page.goto(singleProductPageBaseUrl);
          await page.waitForURL(singleProductPageBaseUrl);
        },
      },
      "cartDrawer.visible": async () => {
        await page.waitForSelector("#cart-notification");
        const cartDrawer = await page.$("#cart-notification");
        if (!cartDrawer) throw new Error("Cart drawer not found");
        const isActive = cartDrawer.evaluate((node) =>
          node.classList.contains("active")
        );
        expect(isActive).toBeTruthy();
      },
      addedToCart: async () => {
        const cartItem = await page.$("#cart-notification-product");
        if (!cartItem) throw new Error("Cart item not found");
        const titleText = await cartItem.evaluate(async (node) => {
          const title = await node.querySelector("h3");
          if (!title) throw new Error("Cart item title not found");
          const titleText = title.textContent;
          return titleText;
        });
        expect(titleText).toBe("The 3P Fulfilled Snowboard");
      },
      inCheckout: {
        meta: {
          urlRegex: new RegExp(
            `^${baseShopUrl.replace(
              /[-\/\^$*+?.()|[\]{}]/g,
              "\\$&"
            )}/checkouts/.*$`,
            "i"
          ),
          waitFor: true,
          expectUrl: true,
        },
      },
      hidden: {},
    },
    events: {
      ADD_TO_CART: async () => {
        const addToCartButton = await page.$('button[name="add"]');
        if (!addToCartButton) throw new Error("Add to cart button not found");
        await addToCartButton.click();
      },
      GO_TO_CHECKOUT: async () => {
        const goToCheckoutButton = await page.$('button[name="checkout"]');
        if (!goToCheckoutButton)
          throw new Error("Go to checkout button not found");
        await goToCheckoutButton.click();
      },
      BUY_NOW: async () => {
        const buyNowButton = await page.$(
          ".shopify-payment-button__button.shopify-payment-button__button--unbranded"
        );
        if (!buyNowButton) throw new Error("Buy now button not found");
        await buyNowButton.click();
      },
      SHOW_CART: async () => {
        const cartButton = await page.$("#cart-notification-button");
        if (!cartButton) throw new Error("Cart button not found");
        await cartButton.click();
      },
      CLICK_ON_VIEW_CART_PAGE_BUTTON: async () => {
        const viewCartPageButton = await page.$("#cart-notification-button");
        if (!viewCartPageButton)
          throw new Error("View cart page button not found");
        await viewCartPageButton.click();
      },
      CLICK_ON_HIDE_CART: async () => {
        const hideCartButton = await page.$(".cart-notification__close");
        if (!hideCartButton) throw new Error("Hide cart button not found");
        await hideCartButton.click();
      },
      CLICK_ON_CART_ICON: async () => {
        const cartIcon = await page.$("#cart-icon-bubble");
        if (!cartIcon) throw new Error("Cart icon not found");
        await cartIcon.click();
      },
    },
  };
}

With some tiny tricks, we can be in pure declarative, trying to further abstract the playwright's logic and start some tests (this is a bit boring and not essential), by using some metadata.

import { test } from "@playwright/test";
import { createTestMachine, createTestModel } from "@xstate/test";

const isAPlaywrightStateFunction = (
  state: PlaywrightState
): state is PlaywrightFunctionState => {
  return isFunction(state);
};
const generateXstateTestStateFromPlaywrightState = (
  playwrightState: PlaywrightState,
  page: Page
): any => {
  if (isAPlaywrightStateFunction(playwrightState)) {
    return async () => {
      await playwrightState();
    };
  }
  const meta = playwrightState.meta;
  return async () => {
    if (playwrightState?.decoratedHandler) {
      await playwrightState.decoratedHandler();
    }
    if (meta?.waitFor && meta.urlRegex) {
      await page.waitForURL(meta.urlRegex);
    }
    if (meta?.expectUrl) {
      expect(page.url()).toMatch(meta.urlRegex);
    }
  };
};
const generateXstateStateAndEventsFromPlaywrightStatesAndEvents = (
  playwrightStatesAndEvents: StateAndEvents,
  page: Page
) => {
  const states = {};
  const events = {};
  Object.keys(playwrightStatesAndEvents.states).forEach((key) => {
    const playwrightState = Reflect.get(playwrightStatesAndEvents.states, key);
    Reflect.set(
      states,
      key,
      generateXstateTestStateFromPlaywrightState(playwrightState, page)
    );
  });
  Object.keys(playwrightStatesAndEvents.events).forEach((key) => {
    const playwrightEvent = Reflect.get(playwrightStatesAndEvents.events, key);
    Reflect.set(events, key, playwrightEvent);
  });
  return {
    states,
    events,
  };
};

createTestModel(advancedAddToCartMachine)
  .getSimplePaths()
  .forEach((path) => {
    test(path.description, async ({ page }) => {
      await path.test(
        generateXstateStateAndEventsFromPlaywrightStatesAndEvents(
          advancedAddToCartMachinePlaywrightStatesAndEventsFactory(page),
          page
        )
      );
    });
  });

This gives us five paths to be tested:

CleanShot 2023 10 18 at 13.40.39@2x — Model-Based Testing for a Secure Shopify Theme with XState 7

Shortest Paths vs Simplest Paths?

Shortest Paths

The shortest path between two points (or states) without passing through the same point twice.
Often used to identify the fastest or most efficient path.
Does not necessarily cover all test cases or features.

Simplest Paths

Any path that passes from a starting point to an end point without revisiting any intermediate point.
Unlike “shortest path”, it may not be the shortest path, but it ensures that no node is visited more than once.
Can cover more features or test cases than the “shortest path”, as it can take longer or different routes.

Translated with www.DeepL.com/Translator (free version)

Results

The model automatically generated five tests for us with a consistent workflow 🤯

Conclusion

Using a finite state machine and playwright, we generated e2e tests by targeting the key steps of a user journey and creating the various associated paths.

Benefits:

Self-generated test (even if GitHub copilot works fine as usual)
Ultra-fast maintenance: if you want to add a step along the way, the machine can be changed very quickly.
High Test Coverage
Easy to automate
More reliable tests
Explicit docs
Helps you to shine in society 🤠

Cons:

It hurts to learn

October 17, 2023 - 12:10 pm

Related posts

blog post thumbnail

Development

Finite State Machines in E-commerce by Example

06 min read

blog post thumbnail

Development

Implementing an effective billing system using Finite State Machines

06 min read

Do you like my work?

Let's make a wonderful
websiteexperienceappstrategy together!

Get in touch!