Krow/Krow-workspace
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
52bb1d1af44accbc5e6f121f0bbdcfff79891607
Krow-workspace/docs/research/maestro-test-run-instructions.md
Suriya 91cedf54c9 testcases
2026-03-06 17:19:12 +05:30

9.8 KiB
Raw Blame History

How to Run Maestro Integration Tests

Credentials are injected via env variables — never hardcoded in YAML.

E2E Happy Path test cases (purpose, steps, expected outcomes): see docs/testing/maestro-e2e-happy-paths.md (#572). Maestro conventions / standards: see docs/testing/maestro-e2e-conventions.md.

Env variables

Flow Env variables
Client sign-in TEST_CLIENT_EMAIL, TEST_CLIENT_PASSWORD
Client sign-up TEST_CLIENT_EMAIL, TEST_CLIENT_PASSWORD, TEST_CLIENT_COMPANY
Client sign_in_invalid_password TEST_CLIENT_EMAIL, TEST_CLIENT_INVALID_PASSWORD (default: wrongpass)
Staff sign-in TEST_STAFF_PHONE, TEST_STAFF_OTP
Staff sign-up TEST_STAFF_SIGNUP_PHONE, TEST_STAFF_OTP
Staff sign_in_invalid_otp TEST_STAFF_PHONE, TEST_STAFF_INVALID_OTP (default: 000000)

Sign-in credentials:

App Email/Phone Password/OTP
Client testclient@gmail.com testclient!
Staff +1 555-555-1234 (use 5555551234 in env) 123123

Step-by-step

1. Install Maestro CLI

Windows: Download from Maestro releases, extract, add to PATH.

macOS/Linux:

curl -Ls "https://get.maestro.mobile.dev" | bash

2. Add Firebase test phone (Staff app)

Firebase Console → Authentication → Sign-in method → Phone → Phone numbers for testing:

  • Add +1 555-555-1234 with verification code 123123

3. Build and install apps

make mobile-client-build PLATFORM=apk MODE=debug
adb install apps/mobile/apps/client/build/app/outputs/flutter-apk/app-debug.apk

make mobile-staff-build PLATFORM=apk MODE=debug
adb install apps/mobile/apps/staff/build/app/outputs/flutter-apk/app-debug.apk

4. Run E2E tests via Makefile

Export credentials, then run:

# Client login credentials
export TEST_CLIENT_EMAIL=testclient@gmail.com
export TEST_CLIENT_PASSWORD=testclient!
export TEST_CLIENT_COMPANY="Test Company"

# Staff login credentials
export TEST_STAFF_PHONE=5555551234
export TEST_STAFF_OTP=123123
export TEST_STAFF_SIGNUP_PHONE=5555550000   # use a new number for signup

# Run full suite
make test-e2e

# Run CLIENT only (auth flows)
make test-e2e-client

# Run CLIENT extended (auth + navigation + orders + settings)
make test-e2e-client-extended

# Run STAFF only (auth flows)
make test-e2e-staff

# Run STAFF extended (auth + navigation + profile + compliance + shifts + benefits)
make test-e2e-staff-extended

5. Run Client only (Windows PowerShell)

$env:TEST_CLIENT_EMAIL = "testclient@gmail.com"
$env:TEST_CLIENT_PASSWORD = "testclient!"
$env:TEST_CLIENT_COMPANY = "Test Company"

# Auth only
make test-e2e-client

# Extended (auth + navigation + orders + settings)
make test-e2e-client-extended

6. Run Staff only (Windows PowerShell)

$env:TEST_STAFF_PHONE = "5555551234"
$env:TEST_STAFF_OTP = "123123"
$env:TEST_STAFF_SIGNUP_PHONE = "5555550000"

# Auth only
make test-e2e-staff

# Extended (auth + navigation + profile + compliance + shifts + benefits)
make test-e2e-staff-extended

Folder structure

apps/mobile/apps/client/maestro/
  auth/
    sign_in.yaml
    sign_up.yaml
    sign_out.yaml
    sign_in_invalid_password.yaml
  navigation/
    home.yaml
    orders.yaml
    billing.yaml
    coverage.yaml
    reports.yaml
  orders/
    view_orders.yaml
    completed_no_edit_icon.yaml  # #492
    create_order_entry.yaml
  settings/
    settings_page.yaml
    edit_profile.yaml
apps/mobile/apps/staff/maestro/
  auth/
    sign_in.yaml
    sign_up.yaml
    sign_out.yaml
    sign_in_invalid_otp.yaml
  navigation/
    home.yaml
    shifts.yaml
    profile.yaml
    payments.yaml
    clock_in.yaml
  profile/
    personal_info.yaml
    documents_list.yaml
    certificates_list.yaml
    time_card.yaml
    bank_account.yaml
    faqs.yaml
    privacy_security.yaml
    emergency_contact.yaml
  compliance/
    document_upload_banner.yaml   # #550
    certificate_upload_banner.yaml # #551
    attire_upload_banner.yaml     # #552
  shifts/
    find_shifts.yaml
    incomplete_profile_banner.yaml # #549 (requires incomplete-profile user)
  home/
    benefits.yaml                 # #524

Direct Maestro commands

To run flows like sign_in with explicit paths and env vars:

# Single flow
maestro test apps/mobile/apps/staff/maestro/auth/sign_in.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123

# Multiple flows (use --shard-split=1; if tcp:7001 persists, run as 2 commands)
maestro test --shard-split=1 apps/mobile/apps/staff/maestro/auth/sign_in.yaml apps/mobile/apps/staff/maestro/compliance/document_upload_banner.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123

# Workaround: run sign_in first, then the feature flow (avoids tcp:7001 between flows)
maestro test apps/mobile/apps/staff/maestro/auth/sign_in.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
maestro test apps/mobile/apps/staff/maestro/compliance/document_upload_banner.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123

Make targets

Target Description
make test-e2e Auth flows (sign_in, sign_up for both apps)
make test-e2e-client Client auth flows
make test-e2e-client-extended Client full suite (auth + nav + orders + settings)
make test-e2e-client-smoke Client smoke suite (deterministic)
make test-e2e-client-orders-smoke Client orders smoke (RAPID → One-Time draft)
make test-e2e-client-hubs-e2e Client hubs E2E (manage + create/edit/delete)
make test-e2e-client-billing-smoke Client billing smoke (overview + invoice details/empty state)
make test-e2e-client-reports-smoke Client reports smoke (dashboard + export placeholder)
make test-e2e-client-settings-e2e Client settings E2E (edit profile save + logout)
make test-e2e-client-orders-data Client orders data-dependent (edit active order)
make test-e2e-client-happy-path Client happy path (auth + hubs + create order E2E + billing + reports + logout) — #572
make test-e2e-client-auth Client auth (sign_in, sign_up)
make test-e2e-client-navigation Client navigation (sign_in + home, orders, billing, coverage, reports)
make test-e2e-client-orders Client orders (sign_in + view_orders, completed_no_edit_icon, create_order_entry)
make test-e2e-client-settings Client settings (sign_in + settings_page, edit_profile)
make test-e2e-client-sign-out Client sign out flow
make test-e2e-staff Staff auth flows
make test-e2e-staff-extended Staff full suite
make test-e2e-staff-smoke Staff smoke suite (deterministic)
make test-e2e-staff-profile-smoke Staff profile smoke (timecard/bank/tax/attire validation)
make test-e2e-staff-payments-smoke Staff payments smoke (earnings history)
make test-e2e-staff-shifts-smoke Staff shifts smoke (find shifts; optionally apply)
make test-e2e-staff-compliance-e2e Staff compliance E2E (document + certificate uploads)
make test-e2e-staff-happy-path Staff happy path (auth + clock in/out + availability + document upload + payments + sign out) — #572
make test-e2e-staff-auth Staff auth (sign_in, sign_up)
make test-e2e-staff-navigation Staff navigation (sign_in + home, shifts, profile, payments, clock_in)
make test-e2e-staff-profile Staff profile (sign_in + personal_info, documents_list, certificates_list)
make test-e2e-staff-profile-extended Staff profile + time_card, bank_account
make test-e2e-staff-sign-out Staff sign out flow
make test-e2e-staff-shifts Staff shifts (sign_in + find_shifts, incomplete_profile_banner)
make test-e2e-staff-compliance Staff compliance (sign_in + document/certificate/attire upload banners)
make test-e2e-staff-home Staff home (sign_in + benefits)
make test-e2e-extended Full suite (both apps, auth + all feature flows)

Troubleshooting

tcp:7001: closed or device offline when running flows

  1. Restart ADB before running: adb kill-server && adb start-server
  2. Try --shard-split=1 when running multiple flows: maestro test --shard-split=1 flow1.yaml flow2.yaml -e ...
  3. If the error persists between flows, run each flow as a separate command. The app stays logged in between runs (Firebase Auth persists): 
    maestro test apps/mobile/apps/staff/maestro/auth/sign_in.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
maestro test apps/mobile/apps/staff/maestro/compliance/document_upload_banner.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123

Client and Staff flows use sign_in-first pattern

All flows assume auth/sign_in.yaml runs first (via Make targets). Flows use launchApp and expect the app to be already logged in. Example:

maestro test apps/mobile/apps/staff/maestro/auth/sign_in.yaml apps/mobile/apps/staff/maestro/compliance/document_upload_banner.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123

Checklist

  • Maestro CLI installed
  • Firebase test phone +1 555-555-1234 / 123123 added
  • Client & Staff apps built and installed
  • Env vars exported
  • make test-e2e-client or make test-e2e-staff run from project root

GitHub Actions

A .github/workflows/maestro-e2e.yml workflow runs Maestro E2E on:

  • Manual trigger (workflow_dispatch)
  • PR/push when maestro flows change

Required secrets (Repository Settings → Secrets):

  • TEST_STAFF_PHONE, TEST_STAFF_OTP, TEST_STAFF_SIGNUP_PHONE
  • TEST_CLIENT_EMAIL, TEST_CLIENT_PASSWORD, TEST_CLIENT_COMPANY

The workflow builds both APKs, starts an Android emulator, installs the apps, and runs auth flows.

Reference in New Issue View Git Blame Copy Permalink