Krow/Krow-workspace
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
91cedf54c9102be3e1358c6882282d1107dc0175
Krow-workspace/docs/research/maestro-test-run-instructions.md
Suriya 91cedf54c9 testcases
2026-03-06 17:19:12 +05:30

268 lines
9.8 KiB
Markdown
Raw Blame History

# How to Run Maestro Integration Tests
Credentials are injected via env variables — **never hardcoded** in YAML.
**E2E Happy Path test cases (purpose, steps, expected outcomes):** see [docs/testing/maestro-e2e-happy-paths.md](../testing/maestro-e2e-happy-paths.md) (#572).
**Maestro conventions / standards:** see [docs/testing/maestro-e2e-conventions.md](../testing/maestro-e2e-conventions.md).
## Env variables
| Flow | Env variables |
|------|---------------|
| **Client sign-in** | `TEST_CLIENT_EMAIL`, `TEST_CLIENT_PASSWORD` |
| **Client sign-up** | `TEST_CLIENT_EMAIL`, `TEST_CLIENT_PASSWORD`, `TEST_CLIENT_COMPANY` |
| **Client sign_in_invalid_password** | `TEST_CLIENT_EMAIL`, `TEST_CLIENT_INVALID_PASSWORD` (default: wrongpass) |
| **Staff sign-in** | `TEST_STAFF_PHONE`, `TEST_STAFF_OTP` |
| **Staff sign-up** | `TEST_STAFF_SIGNUP_PHONE`, `TEST_STAFF_OTP` |
| **Staff sign_in_invalid_otp** | `TEST_STAFF_PHONE`, `TEST_STAFF_INVALID_OTP` (default: 000000) |
**Sign-in credentials:**
| App | Email/Phone | Password/OTP |
|-----|-------------|--------------|
| **Client** | testclient@gmail.com | testclient! |
| **Staff** | +1 555-555-1234 (use `5555551234` in env) | 123123 |
---
## Step-by-step
### 1. Install Maestro CLI
**Windows:** Download from [Maestro releases](https://github.com/mobile-dev-inc/maestro/releases), extract, add to PATH.
**macOS/Linux:**
```bash
curl -Ls "https://get.maestro.mobile.dev" | bash
```
### 2. Add Firebase test phone (Staff app)
Firebase Console → **Authentication** → **Sign-in method** → **Phone** → **Phone numbers for testing**:
- Add **+1 555-555-1234** with verification code **123123**
### 3. Build and install apps
```bash
make mobile-client-build PLATFORM=apk MODE=debug
adb install apps/mobile/apps/client/build/app/outputs/flutter-apk/app-debug.apk
make mobile-staff-build PLATFORM=apk MODE=debug
adb install apps/mobile/apps/staff/build/app/outputs/flutter-apk/app-debug.apk
```
### 4. Run E2E tests via Makefile
**Export credentials, then run:**
```bash
# Client login credentials
export TEST_CLIENT_EMAIL=testclient@gmail.com
export TEST_CLIENT_PASSWORD=testclient!
export TEST_CLIENT_COMPANY="Test Company"
# Staff login credentials
export TEST_STAFF_PHONE=5555551234
export TEST_STAFF_OTP=123123
export TEST_STAFF_SIGNUP_PHONE=5555550000 # use a new number for signup
# Run full suite
make test-e2e
# Run CLIENT only (auth flows)
make test-e2e-client
# Run CLIENT extended (auth + navigation + orders + settings)
make test-e2e-client-extended
# Run STAFF only (auth flows)
make test-e2e-staff
# Run STAFF extended (auth + navigation + profile + compliance + shifts + benefits)
make test-e2e-staff-extended
```
### 5. Run Client only (Windows PowerShell)
```powershell
$env:TEST_CLIENT_EMAIL = "testclient@gmail.com"
$env:TEST_CLIENT_PASSWORD = "testclient!"
$env:TEST_CLIENT_COMPANY = "Test Company"
# Auth only
make test-e2e-client
# Extended (auth + navigation + orders + settings)
make test-e2e-client-extended
```
### 6. Run Staff only (Windows PowerShell)
```powershell
$env:TEST_STAFF_PHONE = "5555551234"
$env:TEST_STAFF_OTP = "123123"
$env:TEST_STAFF_SIGNUP_PHONE = "5555550000"
# Auth only
make test-e2e-staff
# Extended (auth + navigation + profile + compliance + shifts + benefits)
make test-e2e-staff-extended
```
---
## Folder structure
```
apps/mobile/apps/client/maestro/
auth/
sign_in.yaml
sign_up.yaml
sign_out.yaml
sign_in_invalid_password.yaml
navigation/
home.yaml
orders.yaml
billing.yaml
coverage.yaml
reports.yaml
orders/
view_orders.yaml
completed_no_edit_icon.yaml # #492
create_order_entry.yaml
settings/
settings_page.yaml
edit_profile.yaml
apps/mobile/apps/staff/maestro/
auth/
sign_in.yaml
sign_up.yaml
sign_out.yaml
sign_in_invalid_otp.yaml
navigation/
home.yaml
shifts.yaml
profile.yaml
payments.yaml
clock_in.yaml
profile/
personal_info.yaml
documents_list.yaml
certificates_list.yaml
time_card.yaml
bank_account.yaml
faqs.yaml
privacy_security.yaml
emergency_contact.yaml
compliance/
document_upload_banner.yaml # #550
certificate_upload_banner.yaml # #551
attire_upload_banner.yaml # #552
shifts/
find_shifts.yaml
incomplete_profile_banner.yaml # #549 (requires incomplete-profile user)
home/
benefits.yaml # #524
```
### Direct Maestro commands
To run flows like `sign_in` with explicit paths and env vars:
```bash
# Single flow
maestro test apps/mobile/apps/staff/maestro/auth/sign_in.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
# Multiple flows (use --shard-split=1; if tcp:7001 persists, run as 2 commands)
maestro test --shard-split=1 apps/mobile/apps/staff/maestro/auth/sign_in.yaml apps/mobile/apps/staff/maestro/compliance/document_upload_banner.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
# Workaround: run sign_in first, then the feature flow (avoids tcp:7001 between flows)
maestro test apps/mobile/apps/staff/maestro/auth/sign_in.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
maestro test apps/mobile/apps/staff/maestro/compliance/document_upload_banner.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
```
### Make targets
| Target | Description |
|--------|-------------|
| `make test-e2e` | Auth flows (sign_in, sign_up for both apps) |
| `make test-e2e-client` | Client auth flows |
| `make test-e2e-client-extended` | Client full suite (auth + nav + orders + settings) |
| `make test-e2e-client-smoke` | Client smoke suite (deterministic) |
| `make test-e2e-client-orders-smoke` | Client orders smoke (RAPID → One-Time draft) |
| `make test-e2e-client-hubs-e2e` | Client hubs E2E (manage + create/edit/delete) |
| `make test-e2e-client-billing-smoke` | Client billing smoke (overview + invoice details/empty state) |
| `make test-e2e-client-reports-smoke` | Client reports smoke (dashboard + export placeholder) |
| `make test-e2e-client-settings-e2e` | Client settings E2E (edit profile save + logout) |
| `make test-e2e-client-orders-data` | Client orders data-dependent (edit active order) |
| `make test-e2e-client-happy-path` | Client happy path (auth + hubs + create order E2E + billing + reports + logout) — #572 |
| `make test-e2e-client-auth` | Client auth (sign_in, sign_up) |
| `make test-e2e-client-navigation` | Client navigation (sign_in + home, orders, billing, coverage, reports) |
| `make test-e2e-client-orders` | Client orders (sign_in + view_orders, completed_no_edit_icon, create_order_entry) |
| `make test-e2e-client-settings` | Client settings (sign_in + settings_page, edit_profile) |
| `make test-e2e-client-sign-out` | Client sign out flow |
| `make test-e2e-staff` | Staff auth flows |
| `make test-e2e-staff-extended` | Staff full suite |
| `make test-e2e-staff-smoke` | Staff smoke suite (deterministic) |
| `make test-e2e-staff-profile-smoke` | Staff profile smoke (timecard/bank/tax/attire validation) |
| `make test-e2e-staff-payments-smoke` | Staff payments smoke (earnings history) |
| `make test-e2e-staff-shifts-smoke` | Staff shifts smoke (find shifts; optionally apply) |
| `make test-e2e-staff-compliance-e2e` | Staff compliance E2E (document + certificate uploads) |
| `make test-e2e-staff-happy-path` | Staff happy path (auth + clock in/out + availability + document upload + payments + sign out) — #572 |
| `make test-e2e-staff-auth` | Staff auth (sign_in, sign_up) |
| `make test-e2e-staff-navigation` | Staff navigation (sign_in + home, shifts, profile, payments, clock_in) |
| `make test-e2e-staff-profile` | Staff profile (sign_in + personal_info, documents_list, certificates_list) |
| `make test-e2e-staff-profile-extended` | Staff profile + time_card, bank_account |
| `make test-e2e-staff-sign-out` | Staff sign out flow |
| `make test-e2e-staff-shifts` | Staff shifts (sign_in + find_shifts, incomplete_profile_banner) |
| `make test-e2e-staff-compliance` | Staff compliance (sign_in + document/certificate/attire upload banners) |
| `make test-e2e-staff-home` | Staff home (sign_in + benefits) |
| `make test-e2e-extended` | Full suite (both apps, auth + all feature flows) |
---
## Troubleshooting
**`tcp:7001: closed` or `device offline` when running flows**
1. Restart ADB before running: `adb kill-server && adb start-server`
2. Try `--shard-split=1` when running multiple flows: `maestro test --shard-split=1 flow1.yaml flow2.yaml -e ...`
3. If the error persists between flows, run each flow as a **separate command**. The app stays logged in between runs (Firebase Auth persists):
```powershell
maestro test apps/mobile/apps/staff/maestro/auth/sign_in.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
maestro test apps/mobile/apps/staff/maestro/compliance/document_upload_banner.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
```
**Client and Staff flows use sign_in-first pattern**
All flows assume `auth/sign_in.yaml` runs first (via Make targets). Flows use `launchApp` and expect the app to be already logged in. Example:
```bash
maestro test apps/mobile/apps/staff/maestro/auth/sign_in.yaml apps/mobile/apps/staff/maestro/compliance/document_upload_banner.yaml -e TEST_STAFF_PHONE=5555551234 -e TEST_STAFF_OTP=123123
```
---
## Checklist
- [ ] Maestro CLI installed
- [ ] Firebase test phone +1 555-555-1234 / 123123 added
- [ ] Client & Staff apps built and installed
- [ ] Env vars exported
- [ ] `make test-e2e-client` or `make test-e2e-staff` run from project root
---
## GitHub Actions
A `.github/workflows/maestro-e2e.yml` workflow runs Maestro E2E on:
- Manual trigger (`workflow_dispatch`)
- PR/push when maestro flows change
**Required secrets** (Repository Settings → Secrets):
- `TEST_STAFF_PHONE`, `TEST_STAFF_OTP`, `TEST_STAFF_SIGNUP_PHONE`
- `TEST_CLIENT_EMAIL`, `TEST_CLIENT_PASSWORD`, `TEST_CLIENT_COMPANY`
The workflow builds both APKs, starts an Android emulator, installs the apps, and runs auth flows.
Reference in New Issue View Git Blame Copy Permalink