https://www.infoq.com/articles/thoughts-on-test-automation-in-agile

Start Small

Balance the cost vs. ROI well and come up with a bare minimum working solution at the start.

Test Automation Backlog

A test automation backlog could contain a prioritized list of items such as:

• Parameterize the test environment for test execution.
• Integrate with Continuous Integration.
• Enhance reporting mechanism.
• Provide an option to attach error logs in notification emails.
• Collect performance metrics for workflow scenarios.
• Add tests to check for concurrent execution of critical test cases.

Tools Are Just the Means And Not The End

If you focus on the big picture, the real goal is to support new development efforts by providing rapid feedback to the team.

It is also important to keep your tests and test data independent of the selected test automation tool as much as possible.

Create Meaningful Tests and Don’t Try to Automate Everything

The most important part of your test solution is the “tests”.

Get It Out Of Your Local Machine

Execution Time Matters

Keep It Green

Precise Reporting

Make It Visible To Everyone

Test Early

Minimum regression sanity test as a start.

Involve users at the beginning.

Test Often

Corresponding scope of regression set up for each updated build.

Test Automatically

Build up automated SIT with auto-select production data/cases.

Automated meaningful test instead of everything. Note: we need manual effort for automated test building up.

To apply 80/20 rule.