Measurement protocol — StoreFront Studio
Status: protocol only; no runs or comparative performance claims are published yet.
What to measure
Export dimensions, clipping and readability across languages, rendering time and output size. Use synthetic app captures or captures you own, a fixed device/background/layout, and the same set of locale strings for each run.
Reproducible procedure
- Record app version/build, date, OS, hardware, available memory, power mode and relevant settings.
- Identify every input by filename and SHA-256 hash. Use only synthetic or permission-cleared data.
- Record the exact actions, model/provider versions and network conditions. Separate local and cloud operations.
- Run a warm-up separately, then at least five measured repetitions with the same inputs. Distinguish cold-start and warm-start runs.
- Keep raw per-run measurements, failures and verification evidence. Report the median and range; never report only the best run.
- Validate output correctness before interpreting speed. Record defects as well as successful outputs.
- When comparing tools, publish their exact versions, equivalent settings and limitations. Do not infer rankings from untested products.
Use RESULT_TEMPLATE.md. Leave unmeasured fields blank or mark them not measured. A proposed test is not a completed benchmark.