How we test, and why we show our work
Benchmarked Tools runs software through the same task and publishes the method with the result. The first test is of ourselves.
By Benchmarked Tools staff1 min read
A review that cannot be repeated is an opinion. Benchmarked Tools publishes tests, and a test has a method a reader can run again.
What a test looks like
We pick a task a real team would do: draft twenty replies to customer comments, schedule a week of posts across three accounts, pull a month of traffic into a report. We run the same task through each tool, with the same inputs, on the same day. We write down how long it took, what it cost, where the tool stopped and asked for something, and where it fell over.
The method goes in the article next to the result. If a tool does well, you can see why. If it does badly, the vendor can see exactly what we did and tell us what we missed, and we will run it again.
Where the disclosure matters most
Our publisher, Good Standing Media, has a stake in some software companies whose tools we may test. A test site that quietly favours its owner's products is the arrangement readers should distrust most. So when one of those tools is in a test, a disclosure sits at the top of the article and the method is published in full. Readers can run the test themselves and check our numbers.
What we will not call a result
One run is one run. We do not report a difference of a few seconds as a finding. We do not publish a score without the numbers behind it. We do not test a tool we could not get working and call that a verdict; we say we could not get it working and what we tried.
Corrections
hello@benchmarkedtools.com. A disputed result gets re-run, and the article says what changed.