Skip to main contentYour First Simulations
We’ve pre-configured starter simulations for each simulator type so you can immediately see how the platform works and what kinds of results you’ll get.
Running Starter Simulations
- Navigate to Simulations: Go to the simulations page from your dashboard
- Filter by Tag: Add “starter simulations” as a tag filter
- Optional: Add specific simulation type filters if you want to focus on particular simulator types
- Select All: Choose all the filtered starter simulations
- Rerun: Click the rerun button to execute all selected simulations
The simulations will run in parallel and typically complete in about 40 seconds.
Reviewing Results
Once your starter simulations complete:
- Explore each simulation type to understand what different simulators test
- Review conversation transcripts to see how multi-turn testing works
- Check evaluation scores to understand how your chatbot performs across different scenarios
- Note failed interactions to identify potential improvement areas
This gives you immediate insight into queries that your chatbot struggles with in different areas.
Your First Benchmark Tests
We’ve created a “starter benchmark” containing single-turn tests derived from errors found in initial simulation runs.
Running the Starter Benchmark
- Navigate to Benchmarks: Go to the benchmarks section
- Find Starter Benchmark: Look for the benchmark named “starter benchmark”
- Run Tests: You can run individual tests or execute all tests in the benchmark
- Review Results: Check the pass/fail status and evaluation scores
This benchmark gives you an immediate starting point for improving the chatbot and gives you the ability to conduct regression testing as you work.
Next Steps
After reviewing your starter simulation results:
- Create Custom Simulations: Design new simulations with specific parameters to test particular scenarios, topics, goals, etc. For example, try creating simulations to thorougly test the chatbot on questions about upcoming 4th of July sales.
- Build Custom Benchmarks: Add interesting test cases from your simulation runs to your own benchmarks for ongoing regression testing.
- Iterate and Improve: Use the insights to refine your chatbot, then test again. As you add new features or see new areas to test, create new simulations. Always ask the Blast team if you need help testing something.
Understanding Your Results
Simulation Scores
- Scores show pass/fail ratios (e.g., “4/6 pass” means 4 out of 6 evaluators passed)
- Click into individual conversations to see detailed evaluation critiques
- Filter by failed tests to focus on problem areas
Benchmark Results
- Each test shows pass/fail status with detailed scoring
- Compare results across different test runs to track improvements
- Use failed tests to identify consistent problem patterns
Getting Help
If you have questions about your results or need guidance on next steps, the Blast support team is available to help you interpret findings and optimize your testing strategy. 
