Software Architecture Books Summary And Highlights -- Part 14 Testing

-

 

Highlights

Beyoncé Rule: If a product experiences outages or other problems as a result of infrastructure changes, but the issue wasn’t surfaced by tests in our Continuous Integration (CI) system, it is not the fault of the infrastructure change.

SWG ch 1 what is software engineering?



Test Goal

If a fault is present in a system, then we want it to fail during testing as quickly as possible. More emphasis should be placed on finding the most severe faults than on finding other faults.


Test writing guideline

  • Control and observe system state
    • record and playback
    • store states in a single place so that it is easy to start a system, subsystem, or component in an arbitrary state for a test,
    • Preprocessor macros, when activated, can expand to state-reporting code or activate probe statements that return or display information, or return control to a testing console.
  • limit nondeterminism
  • Replace real implementation with test mocks using dependency injection or dynamically based on the context

SAP Ch 12 Testability

Failing to keep a test suite deterministic and fast ensures it will become roadblock to productivity.

In a test instead of setting timeout to wait for a result, it is better to poll every few micro seconds to improve determinism

SWG Ch 11 Testing Overview

Presenters and Humble Objects

The humble object pattern:

Split the behaviors into two modules or classes. One of those modules is humble; it contains all the hard-to-test behaviors stripped down to their barest essence. The other module contains all the testable behaviors that were stripped out of the humble object. e.g. Presenter and View pattern; ORM for db

CLA Part V: Architecture

Test api

The test should be considered part of the architecture.

Changes to common system components can cause hundreds, or even thousands, of tests to break. This is known as the Fragile Tests Problem.

Solution: develop test api. Test api should hide the structure of the application from the tests to decouple test from production code.

This API should have superpowers that allow the tests to avoid security constraints, bypass expensive resources (such as databases), and force the system into particular testable states.

CLA Part V: Architecture


Unit test writing guideline

the ideal test is unchanging: after it’s written, it never needs to change unless the requirements of the system under test change.

Test clarity is very important. To achieve that, we need completeness and conciseness. A test is complete when its body contains all of the information a reader needs in order to understand how it arrives at its result. A test is concise when it contains no other distracting or irrelevant information.

Test Behaviors, Not Methods

Structure tests to emphasize behaviors

Test state, not interactions.

Name tests after the behavior being tested

No logic in tests: The lesson is clear: in test code, stick to straight-line code over clever logic, and consider tolerating some duplication when it makes the test more descriptive and meaningful.

Write Clear Failure Messages

little bit of duplication is OK in tests so long as that duplication makes the test simpler and clearer.

Shared values in tests: Use helper to construct shared value allow each test to specify the param it only needs and without having to worry about specifying irrelevant information in the shared object

Shared setups: sometimes can be confusing when some values are buried in setup

Shared helpers and validations: The best validation helper methods assert a single conceptual fact about their inputs, in contrast to general-purpose validation methods that cover a range of conditions.

SWG Ch 12 Unit Testing


Test Doubles

A test double is an object or function that can stand in for a real implementation in a test

How to Decide When to Use a Real Implementation

  1. Execution time:
    1. use real implementation until it is too slow
    2. parallel test
    3. use efficient build system
  2. Determinism
    1. non determinism comes from external service dependency and system clock
  3. Dependency construction
    1. the real imple’s dependency could be very complicated to setup
    2. use same production dependency construction code to reduce complexity

3 Techniques for Using Test Doubles

  1. fake class

    the team that owns the real implementation should write and maintain a fake.

    before writing a fake, a trade-off needs to be made on whether the productivity improvements that will result from the use of the fake outweigh the costs of writing and maintaining it.

    To reduce the number of fakes that need to be maintained, a fake should typically be created only at the root api level

    If a fake’s clients are across multiple languages, then create a single fake service implementation and have tests configure the client libraries to send requests to this fake service.

    fake must have perfect fidelity to the real implementation, but only from the perspective of the test.

    Fake should fail fast for not covered features

  2. Stubbing you specify to the function exactly what values to return. when(...).thenReturn(...)

    1. When to use: when you need a function to return a specific value to get the system under test into a certain state e.g. error

  3. Interaction testing is a way to validate how a function is called without actually calling the implementation of the function.

real implementation > fake > stubbing > interaction testing

SWG Ch 13 Test Doubles


Large tests

Common gaps in unit tests

  1. unfaithful doubles
  2. configuration issues: bug in deployment configuration
  3. Issues that arise under load
  4. Unanticipated behaviors, inputs, and side effects
  5. Emergent behaviors and isolated from real world mess

Large test challenges:

  1. flaky, slow, not scalable
  2. multiple ownership
  3. lack of standardization. different from prod environment

unit test is valuable to short lived code

larger test is valuable to long running code for long-term healths

SWG Ch 14 Large tests

End to end test ownership

To guarantee the health and quality of the end to end tests, avoiding them to be too flaky or big, The best balance I have found is to treat the end-to-end test suite as a shared codebase, but with joint ownership.

MSV Ch 7 Testing

How to reduce the number of large tests

To reduce the number of large end to end test we should focus on a small number of core journeys to test for the whole system. Any functionality not covered in these core journeys needs to be covered in tests that analyze services in isolation from each other. These journeys need to be mutually agreed upon, and jointly owned.

MSV Ch 7 Testing

Who should define and write e2e test

Let service consumer define expectation and write tests for you to test their usage.

MSV Ch 7 Testing

Best usage of e2e test

end to end test is more used to measure the reliability of the production instead of blocking deployment and detect problem.

MSV Ch 7 Testing

Release remediation time vs more e2e test

Sometimes expending the same effort into getting better at remediation of a release can be significantly more beneficial than adding more automated functional tests. In the web operations world, this is often referred to as the trade-off between optimizing for mean time between failures (MTBF) and mean time to repair (MTTR).

MSV Ch 7 Testing


Related Chapters

SAP Ch 12 Testability

CLA Part V: Architecture

MSV Ch 7 Testing

SWG ch 1 what is software engineering?

SWG Ch 11 Testing Overview

SWG Ch 12 Unit Testing

SWG Ch 13 Test Doubles

SWG Ch 14 Large tests

Popular posts from this blog

Does Free Consciousness exist ？

-
Image
I see a study:  Discovery of quantum vibrations in ‘microtubules’ inside brain neurons supports controversial theory of consciousness  shared on social media. Some one comments that this study indicates that human have free consciouness and the freedom comes from quantum process. I disagree with this comments. To oppose this comment clearly, I will raise and answer two questions. First, what does ‘microtubules’ do inside neurons ? Second what is free consciousness ? What does ‘microtubules’ do inside neurons ? The above picture is from Wikipedia. It shows the structure of a neuron and what ‘microtubules’ is in neuron. The way how neuron works is: A neuron collect chemical signals from many other neurons and generate a signal in the left side of the above picture and send the signal to the right side the above picture through the ‘microtubules’ and finally convert that signal to chemical material and emit them. So ‘microtubules’ is more like a cable to transmit signals. And i
Read more

View and Build Government Organization like a Software

-
  Background Recently I read several books on software architecture. I also read some books about politics and government such as The Logic of Governance in China (Zhou, Xueguang)  and America against America (Wang, Huning). I feel software and government do have many similarities. For example, both of them are created to execute tasks based on some stable business logic,  to satisfy human needs, to serve a large user base, have a topological structure etc. Software system is more like a simplified version of government organizations. In science, a theory is usually created by studying some simple model with a lot of  assumptions and then applied to a more complicated context. I’m thinking maybe we can do the same thing from software architecture to government organization.  The goal of this article is to inspire people on different ways of building government organizations based on software development principles and test them in practice. This is more of a collection of my own thoug
Read more

Software Architecture Books Summary And Highlights -- Part 1 Goal, Introduction And Index

-
  Goal And Target Audience This is a summary and highlights of 6 software engineering and architecture related books I recently read. All the 6 books are very heavy and interesting. Reading real contents of them is always recommended. The target audience is experienced software engineers who want to learn more in larger system design and architecture. The goal of this article is to provide summaries of the 6 books as well as brief highlights of interesting or important contents from the 6 books so that the target audience could learn the 6 books’ important ideas quickly and read only important chapters and save time. The highlights and summaries will be divided into several general categories. Each category contains a highlight section and a related chapters section. The related chapters section contains all chapter references which are related to the category’s topic. The highlight section contains a brief summary of interesting or important or enlightening contents from the relat
Read more