AI software is different from regular software
Hiring ai software development companies using your normal vendor checklist is a mistake — AI software has different failure modes (non-determinism, drift, evaluation) than regular software, and the engineering practices are different.
Questions that filter strong vendors
- How do you evaluate model quality before and after deployment?
- What's your approach to prompt and model versioning?
- How do you handle hallucination in production?
- What does your on-call runbook for an AI incident look like?
- Show me a system you've had in production for 12+ months and what's changed.