java - Is it possible to use JMH as a maven validation check? -

i have performance critical piece of code protect maven build step, i.e. jmh run , check performance hasn't degraded local changes.

how can check such degradation using jmh?

i've found few related links:

• perf testing in ci
• c++ ci perf metrics
• chapter: perf in ci
• ci junit , contiperf (uses @required() annotation)

i've achieved automated performance testing before (though not java, , not in ci environment). 1 key point note never run absolute, since machine benchmark running on can vary. bogomips or test-dependent type of reference can used relative comparison. benchmark measured multiple of reference time, upper , lower bounds.

while typically wary of benchmark slowing (degrading), it's important check upper bound well, may indicate unexpected speedup (better hardware support), should indicate per-system/architecture bound needs checked.

i suggest build set of runner options via optionsbuilder , call run on within junit test. while authors recommend against on grounds of not running benchmark in "clean" environment, think effects marginal , irrelevant when comparing against reference run in same environment.

see here trivial example of setting runner manually.

runner.run() ( or in case of single benchmark runner.runsingle()) return collection<runresult> or runresult, assertions can made against.

in order use statistics (see docs here) can extract runresult via runresult.getprimaryresult().getstatistics() , assert against numeric values can extract statistics

... or use isdifferent() method gives option compare 2 benchmark runs within confidence interval (might useful automatically catch outliers in both directions).