maestra testcases
This commit is contained in:
@@ -68,11 +68,17 @@ Semantics(
|
||||
)
|
||||
```
|
||||
|
||||
### Phase 2: Repository Structure
|
||||
Tests will be localized within the respective app directories to maintain modularity:
|
||||
### Phase 2: Repository Structure (Implemented)
|
||||
Maestro flows are co-located with each app:
|
||||
|
||||
* `apps/mobile/apps/client/maestro/`
|
||||
* `apps/mobile/apps/staff/maestro/`
|
||||
* `apps/mobile/apps/client/maestro/login.yaml` — Client login
|
||||
* `apps/mobile/apps/client/maestro/signup.yaml` — Client signup
|
||||
* `apps/mobile/apps/staff/maestro/login.yaml` — Staff login (phone + OTP)
|
||||
* `apps/mobile/apps/staff/maestro/signup.yaml` — Staff signup (phone + OTP)
|
||||
|
||||
Each directory has a README with run instructions.
|
||||
|
||||
**Marionette MCP:** `marionette_flutter` is added to both apps; `MarionetteBinding` is initialized in debug mode. See [marionette-spike-usage.md](marionette-spike-usage.md) for prompts and workflow.
|
||||
|
||||
### Phase 3: CI/CD Integration
|
||||
The Maestro CLI will be added to our **GitHub Actions** workflow to automate quality gates.
|
||||
|
||||
84
docs/research/maestro-test-run-instructions.md
Normal file
84
docs/research/maestro-test-run-instructions.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# How to Run Maestro Integration Tests
|
||||
|
||||
## Credentials
|
||||
|
||||
| Flow | Credentials |
|
||||
|------|-------------|
|
||||
| **Client login** | legendary@krowd.com / Demo2026! |
|
||||
| **Staff login** | 5557654321 / OTP 123456 |
|
||||
| **Client signup** | Env vars: `MAESTRO_CLIENT_EMAIL`, `MAESTRO_CLIENT_PASSWORD`, `MAESTRO_CLIENT_COMPANY` |
|
||||
| **Staff signup** | Env var: `MAESTRO_STAFF_SIGNUP_PHONE` (must be new Firebase test phone) |
|
||||
|
||||
---
|
||||
|
||||
## Step-by-step: Run login tests
|
||||
|
||||
### 1. Install Maestro CLI
|
||||
|
||||
```bash
|
||||
curl -Ls "https://get.maestro.mobile.dev" | bash
|
||||
```
|
||||
|
||||
Or: https://maestro.dev/docs/getting-started/installation
|
||||
|
||||
### 2. Add Firebase test phone (Staff app only)
|
||||
|
||||
In [Firebase Console](https://console.firebase.google.com) → your project → **Authentication** → **Sign-in method** → **Phone** → **Phone numbers for testing**:
|
||||
|
||||
- Add: **+1 5557654321** with verification code **123456**
|
||||
|
||||
### 3. Build and install the apps
|
||||
|
||||
From the **project root**:
|
||||
|
||||
```bash
|
||||
# Client
|
||||
make mobile-client-build PLATFORM=apk MODE=debug
|
||||
adb install apps/mobile/apps/client/build/app/outputs/flutter-apk/app-debug.apk
|
||||
|
||||
# Staff
|
||||
make mobile-staff-build PLATFORM=apk MODE=debug
|
||||
adb install apps/mobile/apps/staff/build/app/outputs/flutter-apk/app-debug.apk
|
||||
```
|
||||
|
||||
Or run the app on a connected device/emulator: `make mobile-client-dev-android DEVICE=<id>` (then Maestro can launch the already-installed app by appId).
|
||||
|
||||
### 4. Run Maestro tests
|
||||
|
||||
From the **project root** (`e:\Krow-google\krow-workforce`):
|
||||
|
||||
```bash
|
||||
# Client login (uses legendary@krowd.com / Demo2026!)
|
||||
maestro test apps/mobile/apps/client/maestro/login.yaml
|
||||
|
||||
# Staff login (uses 5557654321 / OTP 123456)
|
||||
maestro test apps/mobile/apps/staff/maestro/login.yaml
|
||||
```
|
||||
|
||||
### 5. Run signup tests (optional)
|
||||
|
||||
**Client signup** — set env vars first:
|
||||
```bash
|
||||
$env:MAESTRO_CLIENT_EMAIL="newuser@example.com"
|
||||
$env:MAESTRO_CLIENT_PASSWORD="YourPassword123!"
|
||||
$env:MAESTRO_CLIENT_COMPANY="Test Company"
|
||||
maestro test apps/mobile/apps/client/maestro/signup.yaml
|
||||
```
|
||||
|
||||
**Staff signup** — use a new Firebase test phone:
|
||||
```bash
|
||||
# Add +1 555-555-0000 / 123456 in Firebase, then:
|
||||
$env:MAESTRO_STAFF_SIGNUP_PHONE="5555550000"
|
||||
maestro test apps/mobile/apps/staff/maestro/signup.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Maestro CLI installed
|
||||
- [ ] Firebase test phone +1 5557654321 / 123456 added (for staff)
|
||||
- [ ] Client app built and installed
|
||||
- [ ] Staff app built and installed
|
||||
- [ ] Run from project root: `maestro test apps/mobile/apps/client/maestro/login.yaml`
|
||||
- [ ] Run from project root: `maestro test apps/mobile/apps/staff/maestro/login.yaml`
|
||||
58
docs/research/marionette-spike-usage.md
Normal file
58
docs/research/marionette-spike-usage.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# Marionette MCP Spike — Usage Guide
|
||||
|
||||
**Issue:** #533
|
||||
**Purpose:** Document how to run the Marionette MCP spike for auth flows.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Marionette MCP server** — Install globally:
|
||||
```bash
|
||||
dart pub global activate marionette_mcp
|
||||
```
|
||||
|
||||
2. **Add Marionette to Cursor** — In `.cursor/mcp.json` or global config:
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"marionette": {
|
||||
"command": "marionette_mcp",
|
||||
"args": []
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **Run app in debug mode** — The app must be running with VM Service:
|
||||
```bash
|
||||
cd apps/mobile && flutter run -d <device_id>
|
||||
```
|
||||
|
||||
4. **Get VM Service URI** — From the `flutter run` output, copy the `ws://127.0.0.1:XXXX/ws` URI (often shown in the DevTools link).
|
||||
|
||||
## Spike flows (AI agent prompts)
|
||||
|
||||
Use these prompts with the Marionette MCP connected to the running app.
|
||||
|
||||
### Client — Login
|
||||
|
||||
> Connect to the app using the VM Service URI. Navigate to the Get Started screen, tap "Sign In", enter legendary@krowd.com and Demo2026!, then tap "Sign In". Verify we land on the home screen.
|
||||
|
||||
### Client — Sign up
|
||||
|
||||
> Connect to the app. Tap "Create Account", fill in Company, Email, Password (and confirm) with new credentials, then tap "Create Account". Verify we land on the home screen.
|
||||
|
||||
### Staff — Login
|
||||
|
||||
> Connect to the app. Tap "Log In", enter phone number 5557654321, tap "Send Code", enter OTP 123456, tap "Continue". Verify we reach the staff home screen.
|
||||
> (Firebase test phone: +1 555-765-4321 / OTP 123456)
|
||||
|
||||
### Staff — Sign up
|
||||
|
||||
> Connect to the app. Tap "Sign Up", enter a NEW phone number (Firebase test phone), tap "Send Code", enter OTP, tap "Continue". Verify we reach Profile Setup or staff home.
|
||||
|
||||
## Limitations observed (from spike)
|
||||
|
||||
- **Debug only** — Marionette needs the Dart VM Service; does not work with release builds.
|
||||
- **Non-deterministic** — LLM-driven actions can vary in behavior and timing.
|
||||
- **Latency** — Each step involves API roundtrips (~45s+ for full flow vs ~5s for Maestro).
|
||||
- **Best use** — Exploratory testing, live debugging, smoke checks during development.
|
||||
Reference in New Issue
Block a user