DevelopsenseLogo

The First Hurdle Heuristic

There is a testing techique that I often apply. I have recently decided to name it the First Hurdle Heuristic. The basic idea: get the product out of the starter’s blocks, and see how it performs given a relatively easy challenge.

This heuristic can useful when you want to identify problems and risks immediately, or to determine whether a product might not be ready for use or for deeper testing. Sometimes one or several First Hurdle tests add up to what people sometimes call “smoke testing” or “sanity testing”.

(Why would I apply a different name? Because, for me, it evokes a vivid image of an immediate stumble and faceplant. You can keep calling them smoke or sanity tests if you like, and if you insist, I’ll cheerfully adopt that language when we work together.)

Often the First Hurdle Heuristic can be applied to a claim. For instance: “ChatGPT can generate data for testing.” ChatGPT 3.5 tripped by hooking its foot over the first hurdle as it went over. ChatGPT4o didn’t even try to jump; it simply ran headlong into the first hurdle immediately.

You get to decide whether you want to use a product for any purpose you like. If it’s a Large Language Model, you can use all the tricks you might need to get the result you want: RAG, GAN, CoT, “Simon says do it right the first time.”

If it’s something important, it would be a really, really good idea to examine the output for errors. Then you might want to ask how much time you’d like to spend picking it up, dusting it off, and getting the cinders out of its shoes.

28 replies to “The First Hurdle Heuristic”

  1. Thanks for one’s marvelous posting! I genuinely enjoyed reading it, you happen to be a great author.I will make certain to bookmark your blog and will often come back later in life. I want to encourage that you continue your great posts, have a nice morning!|

    Reply
  2. Thank you for the sensible critique. Me & my neighbor were just preparing to do some research on this. We got a grab a book from our local library but I think I learned more from this post. I am very glad to see such wonderful info being shared freely out there.

    Reply
  3. Good post. I learn something totally new and challenging on blogs I stumbleupon everyday. It’s always interesting to read content from other writers and use something from their sites.

    Reply
  4. You really make it appear really easy together with your presentation but I in finding this topic to be actually something which I believe I’d by no means understand. It sort of feels too complex and extremely wide for me. I’m having a look forward to your next put up, I will try to get the grasp of it!

    Reply
  5. A motivating discussion is worth comment. There’s no doubt that that you need to publish more about this issue, it may not be a taboo subject but generally folks don’t discuss these issues. To the next! Kind regards!

    Reply
  6. I blog frequently and I genuinely appreciate your information. This great article has truly peaked my interest. I am going to bookmark your website and keep checking for new details about once a week. I opted in for your Feed as well.

    Reply
  7. I don?t even know how I ended up here, but I thought this post was good. I don’t know who you are but definitely you are going to a famous blogger if you aren’t already 😉 Cheers!

    Reply
  8. The very next time I read a blog, I hope that it won’t fail me just as much as this particular one. After all, Yes, it was my choice to read, however I truly thought you’d have something helpful to talk about. All I hear is a bunch of complaining about something that you could possibly fix if you were not too busy searching for attention.

    Reply
  9. I absolutely love your website.. Very nice colors & theme. Did you build this amazing site yourself? Please reply back as I’m looking to create my own blog and would love to find out where you got this from or exactly what the theme is named. Many thanks.

    Reply
  10. I used to be more than happy to find this net-site.I needed to thanks to your time for this wonderful read!! I undoubtedly having fun with every little little bit of it and I’ve you bookmarked to take a look at new stuff you weblog post.

    Reply
  11. Having read this I thought it was very enlightening. I appreciate you spending some time and energy to put this informative article together. I once again find myself personally spending a significant amount of time both reading and posting comments. But so what, it was still worthwhile!

    Reply
  12. Can I just say what a relief to discover somebody who really knows what they are talking about on the net. You certainly know how to bring an issue to light and make it important. A lot more people really need to check this out and understand this side of the story. I was surprised you are not more popular since you surely possess the gift.

    Reply
  13. I can’t express how much I appreciate the effort the author has put into writing this remarkable piece of content. The clarity of the writing, the depth of analysis, and the wealth of information provided are simply astonishing. Her passion for the subject is obvious, and it has undoubtedly struck a chord with me. Thank you, author, for offering your insights and enriching our lives with this incredible article!

    Reply
  14. I have observed that over the course of developing a relationship with real estate homeowners, you’ll be able to get them to understand that, in each and every real estate purchase, a commission amount is paid. Ultimately, FSBO sellers never “save” the commission. Rather, they try to win the commission through doing the agent’s work. In this, they expend their money along with time to perform, as best they can, the obligations of an representative. Those jobs include uncovering the home through marketing, presenting the home to willing buyers, constructing a sense of buyer desperation in order to induce an offer, arranging home inspections, controlling qualification checks with the mortgage lender, supervising repairs, and facilitating the closing of the deal.

    Reply
  15. Very nice post. I simply stumbled upon your blog and wanted to say that I have truly loved browsing your blog posts. After all I’ll be subscribing for your rss feed and I’m hoping you write once more soon!|

    Reply
  16. Greetings! I know this is kinda off topic however , I’d figured I’d ask. Would you be interested in exchanging links or maybe guest writing a blog article or vice-versa? My blog goes over a lot of the same topics as yours and I believe we could greatly benefit from each other. If you’re interested feel free to shoot me an e-mail. I look forward to hearing from you! Terrific blog by the way!

    Reply
  17. Pretty portion of content. I simply stumbled upon your website and in accession capital to claim that I acquire in fact loved account your blog posts. Anyway I?ll be subscribing in your feeds and even I achievement you get entry to consistently fast.

    Reply

Leave a Comment