testcases
This commit is contained in:
@@ -3,6 +3,7 @@
|
||||
Credentials are injected via env variables — **never hardcoded** in YAML.
|
||||
|
||||
**E2E Happy Path test cases (purpose, steps, expected outcomes):** see [docs/testing/maestro-e2e-happy-paths.md](../testing/maestro-e2e-happy-paths.md) (#572).
|
||||
**Maestro conventions / standards:** see [docs/testing/maestro-e2e-conventions.md](../testing/maestro-e2e-conventions.md).
|
||||
|
||||
## Env variables
|
||||
|
||||
@@ -188,6 +189,13 @@ maestro test apps/mobile/apps/staff/maestro/compliance/document_upload_banner.ya
|
||||
| `make test-e2e` | Auth flows (sign_in, sign_up for both apps) |
|
||||
| `make test-e2e-client` | Client auth flows |
|
||||
| `make test-e2e-client-extended` | Client full suite (auth + nav + orders + settings) |
|
||||
| `make test-e2e-client-smoke` | Client smoke suite (deterministic) |
|
||||
| `make test-e2e-client-orders-smoke` | Client orders smoke (RAPID → One-Time draft) |
|
||||
| `make test-e2e-client-hubs-e2e` | Client hubs E2E (manage + create/edit/delete) |
|
||||
| `make test-e2e-client-billing-smoke` | Client billing smoke (overview + invoice details/empty state) |
|
||||
| `make test-e2e-client-reports-smoke` | Client reports smoke (dashboard + export placeholder) |
|
||||
| `make test-e2e-client-settings-e2e` | Client settings E2E (edit profile save + logout) |
|
||||
| `make test-e2e-client-orders-data` | Client orders data-dependent (edit active order) |
|
||||
| `make test-e2e-client-happy-path` | Client happy path (auth + hubs + create order E2E + billing + reports + logout) — #572 |
|
||||
| `make test-e2e-client-auth` | Client auth (sign_in, sign_up) |
|
||||
| `make test-e2e-client-navigation` | Client navigation (sign_in + home, orders, billing, coverage, reports) |
|
||||
@@ -196,6 +204,11 @@ maestro test apps/mobile/apps/staff/maestro/compliance/document_upload_banner.ya
|
||||
| `make test-e2e-client-sign-out` | Client sign out flow |
|
||||
| `make test-e2e-staff` | Staff auth flows |
|
||||
| `make test-e2e-staff-extended` | Staff full suite |
|
||||
| `make test-e2e-staff-smoke` | Staff smoke suite (deterministic) |
|
||||
| `make test-e2e-staff-profile-smoke` | Staff profile smoke (timecard/bank/tax/attire validation) |
|
||||
| `make test-e2e-staff-payments-smoke` | Staff payments smoke (earnings history) |
|
||||
| `make test-e2e-staff-shifts-smoke` | Staff shifts smoke (find shifts; optionally apply) |
|
||||
| `make test-e2e-staff-compliance-e2e` | Staff compliance E2E (document + certificate uploads) |
|
||||
| `make test-e2e-staff-happy-path` | Staff happy path (auth + clock in/out + availability + document upload + payments + sign out) — #572 |
|
||||
| `make test-e2e-staff-auth` | Staff auth (sign_in, sign_up) |
|
||||
| `make test-e2e-staff-navigation` | Staff navigation (sign_in + home, shifts, profile, payments, clock_in) |
|
||||
|
||||
138
docs/testing/maestro-e2e-conventions.md
Normal file
138
docs/testing/maestro-e2e-conventions.md
Normal file
@@ -0,0 +1,138 @@
|
||||
# Maestro E2E Conventions (Professional Standard)
|
||||
|
||||
This document defines **how we write and maintain Maestro E2E tests** in this repo, so they stay stable, readable, and useful as a regression safety net—**without** relying on fragile patterns.
|
||||
|
||||
Applies to:
|
||||
- Client: `apps/mobile/apps/client/maestro/**`
|
||||
- Staff: `apps/mobile/apps/staff/maestro/**`
|
||||
|
||||
Related:
|
||||
- Happy-path catalog: `docs/testing/maestro-e2e-happy-paths.md`
|
||||
- Run guide + troubleshooting: `docs/research/maestro-test-run-instructions.md`
|
||||
|
||||
---
|
||||
|
||||
## Test taxonomy (what “professional” means here)
|
||||
|
||||
We maintain 3 tiers of flows:
|
||||
|
||||
- **Smoke (deterministic)**: fast, minimal prerequisites, should pass for a standard test account most of the time.
|
||||
- Targets: `make test-e2e-client-smoke`, `make test-e2e-staff-smoke`
|
||||
- **Happy path (business-critical)**: core “golden paths”, may have real-data dependencies (invoices, shifts, etc.).
|
||||
- Targets: `make test-e2e-client-happy-path`, `make test-e2e-staff-happy-path`
|
||||
- **Data-dependent / state-change flows**: require seeded data (pending invoice, scheduled shift, pending compliance doc).
|
||||
- These are valuable but should be explicitly documented as prerequisites.
|
||||
|
||||
Rule of thumb:
|
||||
- If a flow is **time/location/data dependent**, do **not** include it in smoke.
|
||||
|
||||
---
|
||||
|
||||
## File naming & structure
|
||||
|
||||
- **File name**: use verbs + scope + suffix.
|
||||
- Good: `create_hub_e2e.yaml`, `invoice_approval_e2e.yaml`, `tab_bar_roundtrip.yaml`
|
||||
- Avoid: `test1.yaml`, `new.yaml`
|
||||
- **Folders map to product areas**: `auth/`, `navigation/`, `orders/`, `billing/`, `hubs/`, `profile/`, `compliance/`, `shifts/`, `availability/`, `home/`, `payments/`.
|
||||
|
||||
---
|
||||
|
||||
## YAML header standard (required)
|
||||
|
||||
Each flow should start with:
|
||||
|
||||
- A short description: **what** and **why**
|
||||
- **Prerequisites** (if any)
|
||||
- **Run command** including required env vars
|
||||
- `appId: ...`
|
||||
|
||||
Example header shape (keep short):
|
||||
|
||||
```yaml
|
||||
# Client App — E2E: <Flow Name>
|
||||
# Purpose: <why this matters>
|
||||
# Prerequisite: <only if needed>
|
||||
# Run: maestro test auth/sign_in.yaml <this flow> -e ...
|
||||
appId: com.krowwithus.client
|
||||
---
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Selector strategy (stability rules)
|
||||
|
||||
Priority order:
|
||||
|
||||
1. **`id:` selectors** (best)
|
||||
2. **Accessible text labels** that are stable (good)
|
||||
3. **Coordinates / `point:`** (last resort; document why)
|
||||
|
||||
If you must use coordinates:
|
||||
- Prefer *relative* safe coordinates inside a stable container (not global corners).
|
||||
- Add a comment explaining what UI element it targets and why there’s no better selector.
|
||||
|
||||
---
|
||||
|
||||
## Waiting strategy (reduce flakes)
|
||||
|
||||
Use explicit waits around network UI and animations:
|
||||
|
||||
- Prefer `extendedWaitUntil` for content that depends on API/network.
|
||||
- Use `waitForAnimationToEnd` after navigation transitions.
|
||||
- Avoid long fixed sleeps; wait for a **specific UI marker**.
|
||||
|
||||
Guideline timeouts:
|
||||
- 10–15s typical
|
||||
- 20s only when you know the call is slow (uploads, approvals)
|
||||
|
||||
---
|
||||
|
||||
## Assertions (what “comprehensive” should mean)
|
||||
|
||||
Every flow should have:
|
||||
|
||||
- **Entry assertion**: confirms you are on the expected starting screen (e.g., `assertVisible: "Home"`).
|
||||
- **Critical path assertions**: confirm key steps actually happened (not just taps).
|
||||
- **Exit assertion**: confirms the expected final state (success message, returned screen, etc.).
|
||||
|
||||
Avoid “silent passes” where taps happen but no success state is asserted.
|
||||
|
||||
---
|
||||
|
||||
## Data prerequisites (must be explicit)
|
||||
|
||||
If a flow needs data:
|
||||
|
||||
- Document it in the YAML header and in `docs/testing/maestro-e2e-happy-paths.md`.
|
||||
- Prefer using a dedicated “test account” with predictable fixtures.
|
||||
|
||||
Examples in this repo:
|
||||
- Client invoice approval requires “Awaiting Approval” invoices.
|
||||
- Staff clock-in/out requires an active shift and location constraints.
|
||||
- Staff document upload requires pushing `fixture.pdf` to the device.
|
||||
|
||||
---
|
||||
|
||||
## Fixtures & helper scripts
|
||||
|
||||
If a flow needs a file fixture:
|
||||
- Put it alongside the flow (e.g. `compliance/fixture.pdf`).
|
||||
- Provide a script to push it to the emulator/device.
|
||||
|
||||
Current helper scripts:
|
||||
- `apps/mobile/apps/staff/maestro/compliance/push_fixture.sh`
|
||||
- `apps/mobile/apps/staff/maestro/compliance/push_upload_fixture.sh` (compat wrapper)
|
||||
|
||||
---
|
||||
|
||||
## Suite composition rules
|
||||
|
||||
Professional suites should:
|
||||
- Start with a **single** `sign_in.yaml`
|
||||
- Prefer sequential execution: `--shard-split=1` (already defaulted)
|
||||
- Avoid mixing flows that fight over navigation state unless they’re designed to run back-to-back
|
||||
|
||||
Recommended usage:
|
||||
- Developers run **smoke** on local changes.
|
||||
- QA runs **happy path** before release.
|
||||
|
||||
@@ -59,8 +59,11 @@ All client flows assume the app is **not** logged in at start unless noted (e.g.
|
||||
|----|------|----------|-------|-------------------|------------|
|
||||
| C-ORD-1 | **view_orders** | Orders list loads | After sign_in → Orders → list loads | Order list or empty state | `orders/view_orders.yaml` |
|
||||
| C-ORD-2 | **create_order_entry** | Reach create-order entry (type selection) | Home → Create Order | One-Time / Rapid / etc. options visible | `orders/create_order_entry.yaml` |
|
||||
| C-ORD-2b | **create_order_rapid** | Rapid order screen renders | Home → Create Order → Rapid | Rapid order screen + actions visible | `orders/create_order_rapid.yaml` |
|
||||
| C-ORD-2c | **rapid_to_one_time_draft_submit_e2e** | Rapid → parsed One-Time draft (true E2E) | Rapid → pick example message → Send Message | One-Time draft screen visible | `orders/rapid_to_one_time_draft_submit_e2e.yaml` |
|
||||
| C-ORD-3 | **create_order_one_time_e2e** | Full flow: create one-time order | Home → Create Order → One-Time → fill Event Name, Role → Create | Success; back to orders or confirmation | `orders/create_order_one_time_e2e.yaml` |
|
||||
| C-ORD-4 | **completed_no_edit_icon** | Completed orders do not show edit | Open orders → completed order | No edit action on completed order | `orders/completed_no_edit_icon.yaml` (#492) |
|
||||
| C-ORD-5 | **edit_active_order_verify_updated_e2e** | Edit active order + verify update confirmation | Orders/Home → Edit active order → Confirm & Save | “Order Updated!” confirmation visible | `orders/edit_active_order_verify_updated_e2e.yaml` (requires active order) |
|
||||
|
||||
### Hubs
|
||||
|
||||
@@ -68,6 +71,8 @@ All client flows assume the app is **not** logged in at start unless noted (e.g.
|
||||
|----|------|----------|-------|-------------------|------------|
|
||||
| C-HUB-1 | **create_hub_e2e** | Create a clock-in hub and see it in list | Settings (gear) → Clock-In Hubs → Add Hub → name, address → Create | Hubs list visible; new hub present | `hubs/create_hub_e2e.yaml` |
|
||||
| C-HUB-2 | **manage_hubs_from_settings** | Open hubs management from settings | Settings → Clock-In Hubs | Hubs list / management screen | `hubs/manage_hubs_from_settings.yaml` |
|
||||
| C-HUB-3 | **edit_hub_e2e** | Create hub then edit hub name | Hubs → Add Hub → open details → Edit Hub → Save | Hub updated success message | `hubs/edit_hub_e2e.yaml` |
|
||||
| C-HUB-4 | **delete_hub_e2e** | Create hub then delete hub | Hubs → Add Hub → open details → Delete → confirm | Hub deleted success message | `hubs/delete_hub_e2e.yaml` |
|
||||
|
||||
### Billing
|
||||
|
||||
@@ -75,12 +80,14 @@ All client flows assume the app is **not** logged in at start unless noted (e.g.
|
||||
|----|------|----------|-------|-------------------|------------|
|
||||
| C-BIL-1 | **billing_overview** | Billing tab and overview load | Tap Billing | Billing overview / tabs visible | `billing/billing_overview.yaml` |
|
||||
| C-BIL-2 | **invoice_approval_e2e** | Approve a pending invoice | Billing → Awaiting Approval → first invoice → Review & Approve → Approve | Success feedback; back to list | `billing/invoice_approval_e2e.yaml` (requires at least one invoice in "Awaiting Approval") |
|
||||
| C-BIL-3 | **invoice_details_smoke** | Open invoice details or verify empty state | Billing → (Invoice Ready if exists) | Invoice detail actions visible OR empty-state copy visible | `billing/invoice_details_smoke.yaml` |
|
||||
|
||||
### Reports
|
||||
|
||||
| ID | Test | Purpose | Steps | Expected outcome | How to run |
|
||||
|----|------|----------|-------|-------------------|------------|
|
||||
| C-RPT-1 | **reports_dashboard** | Reports dashboard loads | Tap Reports | Reports dashboard visible | `reports/reports_dashboard.yaml` |
|
||||
| C-RPT-2 | **spend_report_export_smoke** | Spend report export placeholder | Reports → Spend Report → Export | “Exporting Spend Report (Placeholder)” shown | `reports/spend_report_export_smoke.yaml` |
|
||||
|
||||
### Settings & Home
|
||||
|
||||
@@ -88,6 +95,7 @@ All client flows assume the app is **not** logged in at start unless noted (e.g.
|
||||
|----|------|----------|-------|-------------------|------------|
|
||||
| C-SET-1 | **settings_page** | Settings screen opens | After sign_in → Settings (gear) | Quick Links, Log Out, etc. visible | `settings/settings_page.yaml` |
|
||||
| C-SET-2 | **edit_profile** | Edit profile screen opens and is usable | Settings → profile edit entry | Profile form visible | `settings/edit_profile.yaml` |
|
||||
| C-SET-2b | **edit_profile_save_e2e** | Edit profile save + re-open verification | Settings → Edit Profile → Save Changes → re-open | “Profile updated successfully” + “QA” visible | `settings/edit_profile_save_e2e.yaml` |
|
||||
| C-SET-3 | **logout_flow** | Log out via Settings gear | Settings (gear) → Log Out → confirm | Create Account / login screen | `settings/logout_flow.yaml` |
|
||||
| C-HOM-1 | **home_dashboard_widgets** | Dashboard widgets and quick actions render | Home → check Actions, Coverage, etc. | Widgets and "Create Order" visible | `home/home_dashboard_widgets.yaml` |
|
||||
| C-HOM-2 | **tab_bar_roundtrip** | All main tabs can be opened in sequence | Home → Orders → Billing → Coverage → Reports → Home | No crash; each tab loads | `home/tab_bar_roundtrip.yaml` |
|
||||
@@ -120,6 +128,7 @@ All client flows assume the app is **not** logged in at start unless noted (e.g.
|
||||
| ID | Test | Purpose | Steps | Expected outcome | How to run |
|
||||
|----|------|----------|-------|-------------------|------------|
|
||||
| S-SHF-1 | **find_shifts** | Shifts list / find shifts loads | Shifts tab → find shifts | Shifts or empty state | `shifts/find_shifts.yaml` |
|
||||
| S-SHF-1b | **find_shifts_apply_smoke** | Find shifts + optionally apply | Shifts → Find Shifts → (APPLY NOW if exists) | Applying dialog OR empty-state visible | `shifts/find_shifts_apply_smoke.yaml` |
|
||||
| S-SHF-2 | **clock_in_e2e** | Clock in to an active shift | Clock In tab → Swipe to Check In (optional attire photo) | Check-in success | `shifts/clock_in_e2e.yaml` (requires active shift today, within range) |
|
||||
| S-SHF-3 | **clock_out_e2e** | Clock out from shift | After clocked in → Clock Out | Clock-out success | `shifts/clock_out_e2e.yaml` (requires clocked-in state) |
|
||||
|
||||
@@ -138,6 +147,10 @@ All client flows assume the app is **not** logged in at start unless noted (e.g.
|
||||
| S-PRF-3 | **certificates_list** | Certificates list opens | Profile → Certificates | Certificates list | `profile/certificates_list.yaml` |
|
||||
| S-PRF-4 | **time_card** | Time card section opens | Profile → Time Card | Time card UI | `profile/time_card.yaml` |
|
||||
| S-PRF-5 | **bank_account** | Bank account section opens | Profile → Bank Account | Bank account UI | `profile/bank_account.yaml` |
|
||||
| S-PRF-5b | **bank_account_fields_smoke** | Bank account fields visible (when add form is shown) | Profile → Bank Account | Routing/Account number labels visible OR added state | `profile/bank_account_fields_smoke.yaml` |
|
||||
| S-PRF-4b | **time_card_detail_smoke** | Timecard summary labels render | Profile → Timecard | Hours Worked + Total Earnings visible | `profile/time_card_detail_smoke.yaml` |
|
||||
| S-PRF-9 | **tax_forms_smoke** | Tax forms screen loads | Profile → Tax Forms | Tax Forms title visible | `profile/tax_forms_smoke.yaml` |
|
||||
| S-PRF-10 | **attire_validation_e2e** | Attire validation checklist | Profile → Attire → Save Attire | Validation checklist appears | `profile/attire_validation_e2e.yaml` |
|
||||
| S-PRF-6 | **faqs** | FAQs open | Profile → FAQs | FAQ content | `profile/faqs.yaml` |
|
||||
| S-PRF-7 | **privacy_security** | Privacy & Security opens | Profile → Privacy & Security | Screen visible | `profile/privacy_security.yaml` |
|
||||
| S-PRF-8 | **emergency_contact** | Emergency contact opens | Profile → Emergency Contact | Form/section visible | `profile/emergency_contact.yaml` |
|
||||
@@ -150,12 +163,14 @@ All client flows assume the app is **not** logged in at start unless noted (e.g.
|
||||
| S-CMP-2 | **certificate_upload_banner** | Certificate upload CTA visible | Navigate to certificates | Upload banner visible | `compliance/certificate_upload_banner.yaml` (#551) |
|
||||
| S-CMP-3 | **attire_upload_banner** | Attire upload CTA visible | Navigate to attire | Upload banner visible | `compliance/attire_upload_banner.yaml` (#552) |
|
||||
| S-CMP-4 | **document_upload_e2e** | Upload a document (full flow) | Open documents → upload (e.g. PDF) | Upload success | `compliance/document_upload_e2e.yaml` |
|
||||
| S-CMP-5 | **certificate_upload_e2e** | Upload a certificate (full flow) | Certificates → Upload Certificate → upload PDF | Success snackbar visible | `compliance/certificate_upload_e2e.yaml` (requires fixture.pdf pushed) |
|
||||
|
||||
### Payments & Home
|
||||
|
||||
| ID | Test | Purpose | Steps | Expected outcome | How to run |
|
||||
|----|------|----------|-------|-------------------|------------|
|
||||
| S-PAY-1 | **payments_view_e2e** | Payments screen loads and is usable | Tap Payments | Payments list or empty state | `payments/payments_view_e2e.yaml` |
|
||||
| S-PAY-2 | **payment_history_smoke** | Payments history section visible | Earnings → scroll to Recent Payments | Recent Payments visible | `payments/payment_history_smoke.yaml` |
|
||||
| S-HOM-1 | **benefits** | Benefits section loads | Home → Benefits | Benefits content visible | `home/benefits.yaml` (#524) |
|
||||
|
||||
---
|
||||
@@ -171,9 +186,21 @@ From **project root** with env vars set:
|
||||
| `make test-e2e` | Auth flows only (Client + Staff sign_in, sign_up, invalid password/OTP) |
|
||||
| `make test-e2e-client` | Client auth (sign_in, sign_up) |
|
||||
| `make test-e2e-client-extended` | Client: auth + navigation + orders + settings |
|
||||
| `make test-e2e-client-smoke` | Client: deterministic smoke suite |
|
||||
| `make test-e2e-client-orders-smoke` | Client: orders smoke (RAPID → One-Time draft) |
|
||||
| `make test-e2e-client-hubs-e2e` | Client: hubs manage + edit + delete |
|
||||
| `make test-e2e-client-billing-smoke` | Client: billing smoke (overview + invoice details/empty state) |
|
||||
| `make test-e2e-client-reports-smoke` | Client: reports smoke (dashboard + export) |
|
||||
| `make test-e2e-client-settings-e2e` | Client: settings E2E (edit profile save + logout) |
|
||||
| `make test-e2e-client-orders-data` | Client: orders data-dependent (edit active order) |
|
||||
| `make test-e2e-client-happy-path` | Client: auth + hubs + create order E2E + billing + reports + logout |
|
||||
| `make test-e2e-staff` | Staff auth (sign_in, sign_up) |
|
||||
| `make test-e2e-staff-extended` | Staff: auth + navigation + profile + compliance + shifts + benefits |
|
||||
| `make test-e2e-staff-smoke` | Staff: deterministic smoke suite |
|
||||
| `make test-e2e-staff-profile-smoke` | Staff: profile smoke (timecard/bank/tax/attire validation) |
|
||||
| `make test-e2e-staff-payments-smoke` | Staff: payments smoke (earnings history) |
|
||||
| `make test-e2e-staff-shifts-smoke` | Staff: shifts smoke (find shifts; optionally apply) |
|
||||
| `make test-e2e-staff-compliance-e2e` | Staff: compliance E2E (document + certificate uploads) |
|
||||
| `make test-e2e-staff-happy-path` | Staff: auth + clock in/out + availability + document upload + payments + sign out |
|
||||
| `make test-e2e-extended` | Both apps: full extended suites |
|
||||
|
||||
@@ -203,8 +230,11 @@ Use `--shard-split=1` to run flows sequentially and reduce "tcp:7001 closed" iss
|
||||
### Prerequisites that can cause failures
|
||||
|
||||
- **invoice_approval_e2e**: Test account must have at least one invoice in "Awaiting Approval". Otherwise the flow may timeout or not find "Review & Approve".
|
||||
- **edit_active_order_verify_updated_e2e**: Test account must have at least one active (OPEN) order. Otherwise the flow may not find an edit action.
|
||||
- **rapid_to_one_time_draft_submit_e2e**: Requires RAPID parsing to return a One-Time draft screen (network/service dependency).
|
||||
- **clock_in_e2e / clock_out_e2e**: Staff user must have an active shift (today, within check-in window) and be within GPS range (or use mock location).
|
||||
- **document_upload_e2e**: May require a fixture file; see `apps/mobile/apps/staff/maestro/compliance/` for scripts/fixtures.
|
||||
- **certificate_upload_e2e**: Requires `fixture.pdf` pushed to the device (same fixture script as document upload).
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user