About Problem Forge

What we did

As a very mundane description of what's here, every month brings twelve new instances of ten classic problems. In addition, there is a Problem of the Month (which will vary wildly in difficulty) with another twelve instances. The usual leaderboard exists to offer motivation, and everything is solvable with standard tools (given enough time). That is, all scored problems are assured to have solutions. In addition, there are one or more bounty problems -- a prize, usually a very modest honorarium, will be awarded to the first eligible person (or team) to submit a correct solution to any of those.

Extraordinary amounts of effort have been expended to make the implied competition as fair as possible.

Solutions are encoded; nothing here requires anyone to upload code or explain how they got their answers. In the same manner, we've been extraordinarily careful to hide how we did everything behind the scenes. Individuals can decide what they want to try and how they want to proceed. Registration is free -- although we reserve the right to start charging fees for any use which goes beyond mere recreational purposes. We show no ads and sell no data. Exactly what we keep about the participants, why, and for how long is in the privacy notice. (As a quick summary, we keep as little as we can.)

Why we did it

There's a saying that great teachers don't teach, they motivate. This challenge site was inspired by many great teaching websites that provide exercises to build greater proficiency in programming languages, data structures, and algorithms. They rock. The collective body of puzzles, tricks, and challenges available online were built up over decades. Most were lovingly crafted by volunteers. Of course, professionals don't always agree on the relative value of such puzzles when compared to the skills they employ every day; yet, for a time, a selection of these atomic challenges mirrored standard interview questions at places like Alphabet, Amazon, Apple, ... (The full list might include that rare corporation or two that doesn't start with the letter A).

These sites have definitely helped professionals to get started, advance, and get ahead.

When it's not about education or money, it can be about artistic skill. Not everyone uses these resources the same way, yet, striving for an elegant solution usually becomes an integral part of playing the game.

BTW, the implied competitions are definitely games. That implies fun. Letting go. Perhaps whimsy or caprice. For anyone who thinks the way we do, there's some joy in figuring out how to express a problem where a Huffman encoding is the answer without making it obvious that a Huffman encoding is the answer.

Hopefully, there's still more joy in untangling those kinds of knots and tricks.

In some ways, explaining why this site exists is just that simple: We love this stuff. All of it. Problem solving for entertainment purposes being a bit of a niche pursuit, we wouldn't expect consensus on that point. Briefly, we do expect grudging temporary acceptance of the following vague claims: These sites are still very valuable (to some of us anyway); many participants still benefit from what they offer; and, until very recently, the competitive aspects of certain other sites regularly motivated many to acquire new skills.

Here's a metaphor you might spend some time with: When someone brings a sports car to the local Track and Field club and starts smashing all the local records, it does tend to be a bit discouraging to the athletes. Maybe there's some detectable difference between owning a Ferrari and having spent years training for marathons. Anyway, there's another possible answer: Maybe we created this arena because sports car enthusiasts need a different kind of competition. (Maybe think Dakar rally, not circling around a well-groomed Formula One track.)

If the metaphor doesn't land well for you, the unfortunate social change we're forced to discuss surrounds a tiny shift in perception surrounding the word motivate. Mildly, the widespread (and growing) proliferation of LLMs has shifted a bit of the landscape in this area. Anyway, that's likely the mildest presentation of some of the controversies raised by LLMs (and all the hype and puffery that surrounds them) that we can manage. Perhaps in other places, the right response to all that kind of noise and drama involves a long, argumentative essay cast into the wind of social media.

That's just not the way we roll here; instead, we've designed a game.

(Yeah, "if all you have is a hammer, ..." Fair criticism. We'll own it.)

You can use the resources here to competitively evaluate LLMs.

You can use the resources here to competitively evaluate prompting skills.

If you're so inclined, you're encouraged to demonstrate what humans can do without the use of LLMs.

Legal note: This is individual benchmarking offered for entertainment purposes only. Formal benchmarking for any other purpose may be offered in the future, but is highly discouraged during the Beta test.

What might motivate you to participate

We have no idea what motivates you personally. The world is vast and houses many people with different thoughts. There are lumps in the gravy of arguments and it's all very messy. According to a political economist friend, opinions tend to polarize and align around price points. One side wants prices to go up; another wants them to go down. (We didn't fully understand that explanation either.) Anyway, that got us thinking about being referees in a greater tug of war, and we came up with this:

If you have something to prove about AI, this might be the place to do it.

We could speculate a bit about the teams that are forming. Perhaps you believe that, soon, human beings will never have to touch programming languages again. You might have a "my LLM is better than your LLM" insult which demands a response and want to borrow a dueling field.

Or maybe you see prompting as a skill or think of it as just a different form of programming. You might think that tomorrow's successful engineers will need to learn to use the new tools effectively.

More than a few individuals have been suggesting that humanity has to pull the plug on all the horrifying changes while we still can. To be fair, a lot of scenarios played out in literature and films over the last century do wind up with robots in charge (or narrowly defeated by clever humans). So, maybe, just maybe, a few of you want to rehearse that battle early and get your strategies organized. On that note, creating a program by hand which can outperform an artifact produced by AI may or may not be as difficult as you might think. (The harder task might be keeping your research and code out of the great plagiarism engine's vast intake valves.) Or it could all be as simple as wanting to use up some extra tokens that you've already spent money on. After all, there are those bounties to consider. (Our political economist friend made this point too: "Money is good; it can be exchanged for goods and services." Our well-considered response at the very end of a long conversation about money: "True, lottery tickets are a better bet, but where's the fun in that?")

We could talk about all these perspectives endlessly. Let's get some hard data instead.

What might motivate someone to do more

More hand waving about us not being mind readers. If problem solving for entertainment purposes is a bit of a niche pursuit, problem setting for entertainment purposes is a niche-within-a-niche pursuit. Every month, we'll need new material for the Problem of the Month. As with any new project, we need help on smaller things too. Accurate translation being at the top of that list. We're not soliciting volunteers, but we're interested if you're interested. Just send us an email, and we'll set up a time to talk.

Unfortunately, you should never send us ideas. Legal reasons. Any material sent to us is implicitly released to us. Just don't bother. On another level, the plan for future development of the site will open new paths. There'll be a opportunity to post difficult problems and offer bounties. If you can figure out what's really going on here, you might see many reasons far beyond mere entertainment to create puzzles for others to solve.

Finally, you don't get doughnuts from the hardware store. We're obviously not making money. So please don't bother trying to scam us.

Who runs it

Problem Forge is run by Problem Forge, LLC. This is our circus and those are our monkeys.

Contact: admin@problemforge.com.

BTW, the checker is open source (MIT). Problem files and answers are in the public domain (CC0). The rest of the site is proprietary.

Credits

The logo is an anvil drawn by Gerald G and was first shared on Openclipart. We took it from the file "Anvil, labelled en.svg" on Wikimedia Commons, where Andy Dingley added labels. The home page shows it with the labels, translated. The small logo leaves them out and is cropped. The author dedicated the drawing to the public domain under CC0 1.0 Universal. No credit is required, but we give it anyway, and offer our thanks.

Support

Problem Forge is supported by its players and sponsors. See Support From.

Confused?

New here? Read Start here. For additional reading on the classic problems, see References. Thinking of using AI? Read On using AI.