\n<\/p>\n
Follow ZDNET: <\/em>Add us as a favorite source<\/span><\/a><\/span> On Google.<\/em><\/p>\n Here at ZDNET, we know we have an amazing responsibility. We know you often make purchasing decisions based on our reviews. It is important that you find clear, unbiased, well-reasoned reviews so that you have a reliable starting point for deciding where to spend your money and\/or time.<\/p>\n And yes, we take responsibility for free products just as seriously, because time is as scarce a resource as cash these days. We don’t want you to waste your time any more than we want you to waste your money.<\/p>\n Also: ZDNET AI Policy<\/strong><\/p>\n We sometimes work with vendors to gain access to their products and services so they can be reviewed. But they never get to see the reviews before we publish. They never influence what we say in our reviews. Our reviews are always unbiased and focused on assessing the usefulness of the products to our readers.<\/p>\n So let’s talk about how we test AI here at ZDNET. Keep in mind that AI is making its way into almost everything, so this is a pretty big portfolio. We see large language models, development tools, image generators, AI-enabled applications, and even the occasional AI device like the vacuum cleaner (good use of AI) and the AI \u200b\u200bpin (not so much). <\/p>\n We test products and services based on many factors. Our main directive is that all reviews require practical experience and real-world tests. In practical terms, this means that while we may report on benchmark results from a press release, we don’t consider them in reviews. <\/p>\n When we look at products and services, we come across two different types of reviews. When we look for the top performers in a category, we create our \u201cBest of\u201d lists. When we think deeply about a product or service, we often tell personal stories about our long-term experiences using that product. These different viewpoints allow us to explore products and services from many different perspectives. <\/p>\n Creating our comparison reviews (also known as \u201cbest of\u201d lists) is actually a three-step process. The first step is creating evaluation criteria to help us compare products fairly. The second step is to select the products to compare. And the third step is actually a test-by-test comparison of the products. <\/p>\n When we start out, we always ask, “How will we evaluate this category?” I usually set up a series of tests, which I document in a Best of List article. Tests help us evaluate performance, value, usability, accuracy, security, privacy, and more. We like to standardize on testing so that when it’s time to compare products, we know we’re being objective. <\/p>\n For example, the best chatbot reviews have a full testing methodology documented at the end of the product. Check it out. The same is true of the best AI image generator comparison. <\/p>\n When it comes to choosing candidate products, there are often a few obvious products that get added to our selection candidate list. For example, when looking at chatbots, ChatGPT, Gemini, and Cloud are obvious candidates. <\/p>\n Then we delve deeper. We review products or services that readers have asked us to evaluate. We add candidates based on the overall discussion about a category from places like forums, user groups, and social media. And sometimes (but not always), we’ll add a product as a candidate when a seller brings a relevant product to our attention, and it’s a good fit for the category. <\/p>\n We typically end up with a candidate list of five to ten products. Often, a quick look at the testing methodology will eliminate some products. Some are much more expensive than others. Some don’t fit at all. <\/p>\n For example, I’m constantly criticized by vendors of fee-based classes who think their course is so good that it should be included on our list of the best free classes. Despite their enthusiasm, their fee-based courses will never be included in the list of free offerings. <\/p>\n The process of selecting test candidates, arranging access to products and services, and ensuring that everything is ready to run the test can vary over time. When I took my first look at AI website builders last year, it took 231 emails to vendors, and more than six months to get everything set up so I could test their products. This year, it took only two months to update the project, and the total number of emails was less than 50. <\/p>\n This leads me to two other items: actual testing and re-testing. The actual test is straightforward, if time consuming. Since we already have a testing methodology and a standard set of tests, until we have established product or service accounts in hand, we can simply test. We record the results test by test, screen by screen. <\/p>\n Later, we try to normalize the results, often doing a little math to give products comparable performance values \u200b\u200band weights. The criteria for those metrics are also documented. <\/p>\n And then, the list is published. But this is not the end of the story. <\/p>\n In a rapidly changing field like AI, products and services do not remain static. Some products will be destroyed and burned, some vendors will lose funding, or something else will go horribly wrong. For others, they will get better and better. In any case, after six months to a year, best-of lists become quite outdated. This was certainly the case with AI website builder reviews. Last year, they were all terrible. This year, there are some that are really great. <\/p>\n Some of my favorite comparison reviews for the AI \u200b\u200bcategory include: <\/p>\n Another way to review AI products is to live with them and do projects with them. These go beyond traditional reviews because we put days and weeks (sometimes months and years) of work into products and services. <\/p>\n A prime example of this are my coding-related articles. It’s very hard to objectively compare AI coding tools without actually building something. But coding a class assignment is very different from building a product or debugging an active customer issue. <\/p>\n Often these projects remain ongoing. That ongoing work produces lots of great things to talk about. Perceptions also change. <\/p>\n When I first saw OpenAI’s Codex coding AI, it was very early and I didn’t like it at all. As the codecs improved, I did another test with it, this time to see if I could update my security product. I managed to complete 24 days of coding in 12 hours, but I also found some shortcomings. As the service improved further, I did another test where I found I was doing 4 years of product development in 4 days. <\/p>\n Similar experiential review articles have appeared about Gemini, ChatGPT, Cloud Code, various image generators, and others. As tools continue to evolve, we continue to find new ways to use them and test them more thoroughly. <\/p>\n It is an ongoing process and we take you with us on the journey. Here are some of my favorites from the AI \u200b\u200bworld: <\/p>\n We get a lot of feedback from readers through email, social networking and article comments. You help us understand what you want to show us. We also appreciate that you hold us to such a high standard. <\/p>\n We really appreciate it when you share your thoughts about the products we review. Many of you are quite skilled and knowledgeable. So your viewpoints really help keep us informed, which in turn, helps us increase our knowledge and keep you even more informed. Effectively, our work here at ZDNET is reviewed by millions of our fellow professionals, power users and enthusiasts: you, the ZDNET reader. <\/p>\n We’re diligent about our reviews because we know how important they are to you, how much you consider them when making purchasing decisions, and you’re putting real money and time at stake, often based on what we share on ZDNET. <\/p>\n Always feel free to contact us if you want us to see something new. What AI category, product or service would you like us to focus on next? Let us know in the comments below. <\/p>\n You can follow my daily project updates on social media. Be sure to subscribe My weekly update newsletter<\/a>And follow me on Twitter\/X @davidgewirtz<\/a>on facebook Facebook.com\/DavidGewirtz<\/a>on instagram Instagram.com\/DavidGewirtz<\/a>on bluesky @DavidGewirtz.com<\/a>and on youtube YouTube.com\/DavidGewirtzTV<\/a>.<\/em><\/p>\n<\/div>\nZDNET Highlights<\/h3>\n
\n
How will we test AI in 2026 <\/h2>\n
How do we do comparative reviews <\/h2>\n
stay with products <\/h2>\n
You are a big part of the process <\/h2>\n