Introduction au Mutation Testing

Nicolas Hart

Développeur Web chez TER'Informatique

@nclsHart

Mutation Testing

Comment contrôler la qualité de ses tests ?

Couverture de code ?

Insuffisant

Mutation Testing

Principe

Exemple de mutant

public function add($a, $b)
{
-    return $a + $b;
+    return $a - $b;
}

Principe

Mutant tué

• Les tests ne passent pas
• Le code mutant a été détecté par les tests
• Le code muté est correctement testé

Mutant vivant

• Les tests passent
• Le code mutant n'a pas été détecté par les tests
• Le code muté est probablement pas ou mal testé

Mutators

Inversion de condition :

- if ($a === $b) {
+ if ($a !== $b) {

Modification de valeur de retour :

- return $this;
+ return null;

Etc

Métriques

• Mutation Score Indicator (MSI)
• Mutation Code Coverage (MCC)
• Covered Code Mutation Score Indicator (CCMSI)

Cas réel

Utilisation d'Infection

• TheRedDot/swiftmailer-image-embed

Résultat

23 mutations were generated:
      19 mutants were killed
       0 mutants were not covered by tests
       4 covered mutants were not detected
       0 errors were encountered
       0 time outs were encountered

Metrics:
         Mutation Score Indicator (MSI): 82%
         Mutation Code Coverage: 100%
         Covered Code MSI: 82%

Time: 2s. Memory: 12.00MB

Exemple mutant tué

[M] MethodCallRemoval

--- Original
+++ New
@@ @@
 if ($this->fileExists($path)) {
     $entity = \Swift_Image::fromPath($path);
     $message->setChildren(array_merge($message->getChildren(), [$entity]));
-    $image->setAttribute('src', 'cid:' . $entity->getId());
+                    
 }

Exemple mutant en vie

[M] ProtectedVisibility

--- Original
+++ New
@@ @@
-    protected function getPathFromSrc(string $src) : string
+    private function getPathFromSrc(string $src) : string
     {
         if ($this->isUrl($src)) {
             return $src;

Résultat après corrections

19 mutations were generated:
      19 mutants were killed
       0 mutants were not covered by tests
       0 covered mutants were not detected
       0 errors were encountered
       0 time outs were encountered

Metrics:
         Mutation Score Indicator (MSI): 100%
         Mutation Code Coverage: 100%
         Covered Code MSI: 100%

Time: 2s. Memory: 12.00MB

Conclusion

Points positifs

• Facile à mettre en place
• Retour sur les tests et le code

Points négatifs

• Temps d'exécution
• Faux positifs

Merci !