Tried it! I really like the idea, but I think the clue generation could use some work. Every clue ended in "in games", and honestly most of them were not really game related to start with. For example the clue "Place in games where characters go to rest and replenish health or mana" had the solution "bar"... which I wouldn't describe as right. Similarly "The name of a popular character who may need rescuing in some games" was "Emily".
I think it might be worth working on prompting to make sure the answer is a unique solution to the hint (or at least closer to unique). What model are you using here?
emily is the name of characters that are rescued in bioshock infinite as well as the first dishonored game. both games are around a decade old but were popular at the time.
Isn't the character in Bioshock Infinite 'Elizabeth'? I'd also assert that by design, Elizabeth was meant to be a character that arguably didn't really need to be rescued, "she can take care of herself".
The only name I could think of in 5 letters that fit here was actually "Peach".
Thank you for trying.
You are absolutely right.
In my defense i released the very first 20 puzzles without proof reading them. I just wanted to see what the AI can deliver as a starting point and get an idea if crossword players would like or hate the general idea of it. I've just started seriously playing it myself and some clues are indeed strange. The clues were generated by gpt-4o. You can try later puzzles i did some prompt adjustments because i noticed the forced "in games" after 10 puzzles or so. It gets a bit better at later puzzles. Thank you for the feedback
Really goes to show how bad top-tier LLM's are at rather basic tasks like creating a clue out of popular media. This should be among the core competencies of major models with tons of available training data and simple summarization requests.
LLM Hallucinations are real. Admittedly for the prompt I provided just the word and some basic rules. It should be possible to increase the quality if e.g. I provide the sentence in which the word was used to the LLM. Nevertheless Hallucinations will always be a problem i think there is no way around a human quality gate in the process?
Nice work. I've also experimented with procedurally generated crossword puzzles, though I really wanted to constrain them to symmetric layouts like what you would find in the New York times which made it more difficult.
There's an outstanding issue and that is (from what I can tell) at least 75% of the answers correspond to relatively generic nouns or verbs.
Part of the deep satisfaction in solving a crossword puzzle is the specificity of the answer. It's far more gratifying to answer a question with something like "Hawking" then to answer with "scientist", or answering with "mandelbrot" versus "shape".
It might be worth going back and looking up a compendium of games released in the last couple decades, cross referencing them with their manuals, GameFaqs, etc. and peppering this information into the crossword.
Wanted to thank you again. I am currently working on an improved version providing more context. Your sentence "Part of the deep satisfaction in solving ..." made it into the prompt's rule-set. At this very moment I am only using the dataset of r/dota2 to make the testing easier and I look at the very first result with the new prompt:
Generated words and clues:
heroes: Characters with unique abilities in Dota 2, tasked with defeating the enemy's Ancient.
ragers: Players who overly react to in-game frustrations, often ruining the fun for everyone.
rage: A common emotion experienced by players sometimes leading to poor decision-making.
tachyons: Hypothetical particles that travel faster than light, having no place in an Ancient's mechanics.
healing: Essential support function often provided by certain heroes like Treant Protector.
burn: Refers to a mechanism used to deplete an opponent's mana, crucial in trilane strategies.
matters: In Dota 2, every decision, including hero picks, can significantly change the outcome.
fault: What a player will often blame when losing, rather than acknowledging their own mistakes.
support: Role in Dota 2 focused on helping the team, often with abilities to aid and sustain.
team: Group of players working together to win, where synergy and composition are key to victory.
Note that the Words themselves were not picked by OpenAI but rather a per-selection from the BERT Embeddings ML Algorithm but this time with more than just a word as context.
This is definitely going in the right direction. It's only sample size of 1 but i had to share it with you!
Absolutely! Keep me updated - these types of projects are definitely fun to explore. It'll also be interesting to explore alternative LLMs, as well as providing zero shot examples within the system prompt itself (if you haven't already).
I forgot to mention but it might also be worth exploring more classic NLP techniques like named entity recognition to score clues higher and lower in terms of overall specificity.
Have not done zero shot yet. At the moment i am experimenting with this https://arxiv.org/abs/2210.05245. Looks promising hoping to push an updated version soon but i am still fighting with it at the moment
Thank you for your input. Your conceptual point on what makes crosswords fun is gold! Your practical advice is not only great but also very doable: Remember I used a 3.7million Reddit-Comments dataset which is a sub-optimal choice to generate words out of. (But i liked the headline behind it). So changing the dataset to something you mentioned (game compendiums, manuals, faqs) will probably increase the answers significantly.
After a ~30 hours weekend coding marathon, I've just pushed a new version of the original joystick-jargon (r/gaming) and a new r/leagueoflegends puzzle live.
Echoing what vunderba said, it's hard for me to enjoy this because I'm used to the symmetric layouts of e.g. the NYT and the satisfying flavor of their clues.
However, this is well done and it inspired a thought- I wonder if it would be possible to procedurally generate word games, such as a mini crossword or word ladder or so on, as part of a language learning regime? Think Duolingo but for word puzzle fans.
As an example, you solve a mini crossword every day where 80% of the clues/answers are in English, and 20% are drawn from a progressive set of vocabulary in the other language.
Haha, true. I was sure it is "Inn" from Hearthstone Innkeeper but that's a case of LLM Hallucinations. See my other comment on how this can be improved but not solved completely imo
I had to reduce the size in my browser a couple or so times to see both the puzzle and all the clues at the same time. It might be better to have the clues in a separate scrolling region on the page.
The way the NYT does this on their web interface is nice. They have the puzzle in one column, the across clues in a second column, and the down clues in a third column. The clue columns each are scrollable.
It automatically scrolls to keep the clue for whatever word you have selected in view and highlights that clue, and also automatically scrolls to keep the clue for whatever word crosses that word at the particular square you have highlighted is also visible and marked in the margin of its clue list.
They do similar in their iPad app, but also below the puzzle show the clue for the selected word and for whatever word crosses it at the highlighted square. With that you can concentrate on the grid and a fixed clue area.
Thank you for that feedback. I agree NYT does this a lot better. I'll have to improve on that. Not gonna lie to you I suck at CSS/Styling/UX/UI will need to get some help regarding this. However i feel you and it's important for the fun of the game
Please respect your users. It’s frustrating to spend your time on a puzzle that might not even be plausibly solveable. If you’re going to ask strangers to do free testing you should at least bother to do some basic proofreading first.
Learnt my lesson. Truly felt (still) bad and sorry after people were saying they spent time and would not click solve easily. I only looked at it from a technical perspective and fully underestimated how serious the cross-wording community is. There was no bad intention and I will absolutely respect that from now on!
//EDIT. I've been working on it for approx. 10 hours today (still going) with the goal to put live a much better quality version. I feel the pressure and want this fixed myself. However it's quite a computational challenge i am giving my very best
The 13th puzzle is probably the first decent one. I still can't quite put together how it's getting the clues in some cases (BMing = using pedals? noly = Just this, no other?) but it's certainly the first one that's within reach of being reasonable.
TYVM for this feedback. I feel the exactly same way. This is purely a NON-PROOF-READ starting point, a field test of what AI can put out unsupervised. The prompt was changed in the latter puzzles those should be a tad better. But as already discussed there can be done a lot more to improve the quality of words and clues significantly.
I love crossword puzzles, and this is very well done! A while back I made a crossword puzzle generator using constraint programming. It is definitely way slower than whatever heuristic you are using, but it is very good at generating dense crossword puzzles using small word lists. This would be useful if you were interested in making a crossword puzzle using words related to a single game, for instance. You can read more on it here if you are interested: https://pedtsr.ca/2023/generating-crossword-grids-using-cons...
Thank you for sharing. I will check it out. And no worries regarding the speed. The website is cheating the 20 puzzles are pre-compiled so to say, the actually work was done in C++, the web server is just indexing an array of perfect puzzles at that point. TY!
> 1: Allows movement between different areas in a game
> Unportal
what?
>19: An online forum section where gamers come together to discuss specific topics
> ITT
I don't think that's right...
Most of the questions just seem like normal crossword questions, but with the term "in games" added to it.
I'm not gonna sugarcoat it: this sucks. The crossword grids often have totally isolated words. 1 across and 2 down start with the same letter. The questions are nonsensical. I'd hardly go so far as to call it an interesting proof of concept.
Thank you for trying! Please see my reply on the other comment it fits here as well. This is a non proof read version and i agree with you consider this a starting point. Also feel free the try out the later puzzles (10+) i changed the prompt at some point because i noticed the same thing.
Edit: At some point I was unsure myself, if i don't know certain things about gaming or if the AI is making things up, haha, so this feedback is extremely valuable for me thank you
TYVM for trying it out and feedback and it confirms what other users are saying. This is an unsupervised version with no human quality gate. Please consider this a starting point. That being said, I feel bad that you couldn't even enjoy it, that was not my goal. Sorry for that. I thought people would take a smile away regardless of some words/clues being bad. I have to admit i underestimated the appeal of quality. Some people say it gets at least a little bit better from puzzle #13, maybe you can get some enjoyment out of that? No worries if you are fed up at this point. Thanks for trying it out
>5. Grid Generation: Implemented a heuristic crossword algorithm to create grids and place words efficiently.
I always think about doing something similar for a similar project. Are you able to do it completely automatically or do you have to help finesse the words to fit?
Thank you! You right, no i18n yet, I can assure you that the URL https://capsloq.de/crosswords/joystick-jargon has the only playable crosswords at this moment. The other links in the Nav are not crossword related. Will definitely do i18n maybe even the crossword themselves in the future.
Unportal? What the fuck is unportal? I have played games for 30 years and I've never heard the term "unportal". Google gives no useful results. That clue made me angrier than any crossword clue I've ever seen.
TYVM for trying it out! Oh no, now I feel bad again. Please remember these are unsupervised generated clues by a Large Language Model (openai). More of a field test, however I now understand how serious you and others take crosswording. I vow to do better for the next crossword puzzles. It's not my intention to make people feel bad, sorry!
One more thing: Besides the obvious that clues and answers should make sense can you provide some insights in what makes a good crossword to you? I'd love to hear that from someone playing them for 30 years!
For me, a mixed crossword between all games doesn't make sense. You could give me 20 crosswords with Dota stuff in it and I would be able to solve 0, not even with hints it would be doable.
So maybe rather have some for League, some for CS, etc? Maybe you can do a mixed indie one with very popular games, or mixed Shooter. But then the questions have to be less difficult :D
TY for the feedback. I see what you are saying about the crossword getting too niche. For now this is mostly a technical decision since its easier to train on a more specific dataset. However i agree with you that the dataset should be expanded once the quality is there
Tried it! I really like the idea, but I think the clue generation could use some work. Every clue ended in "in games", and honestly most of them were not really game related to start with. For example the clue "Place in games where characters go to rest and replenish health or mana" had the solution "bar"... which I wouldn't describe as right. Similarly "The name of a popular character who may need rescuing in some games" was "Emily".
I think it might be worth working on prompting to make sure the answer is a unique solution to the hint (or at least closer to unique). What model are you using here?
Haha, for those two you mentioned I assumed it was 'Inn' and 'Zelda'. I don't even know who Emily is.
And googling 'Emily video game character' didn't bring up any noticeably popular video game characters.
I put "hub" and "Peach"... There seems to be a lot of possible answers
emily is the name of characters that are rescued in bioshock infinite as well as the first dishonored game. both games are around a decade old but were popular at the time.
Isn't the character in Bioshock Infinite 'Elizabeth'? I'd also assert that by design, Elizabeth was meant to be a character that arguably didn't really need to be rescued, "she can take care of herself".
The only name I could think of in 5 letters that fit here was actually "Peach".
Thank you for trying. You are absolutely right. In my defense i released the very first 20 puzzles without proof reading them. I just wanted to see what the AI can deliver as a starting point and get an idea if crossword players would like or hate the general idea of it. I've just started seriously playing it myself and some clues are indeed strange. The clues were generated by gpt-4o. You can try later puzzles i did some prompt adjustments because i noticed the forced "in games" after 10 puzzles or so. It gets a bit better at later puzzles. Thank you for the feedback
Really goes to show how bad top-tier LLM's are at rather basic tasks like creating a clue out of popular media. This should be among the core competencies of major models with tons of available training data and simple summarization requests.
LLM Hallucinations are real. Admittedly for the prompt I provided just the word and some basic rules. It should be possible to increase the quality if e.g. I provide the sentence in which the word was used to the LLM. Nevertheless Hallucinations will always be a problem i think there is no way around a human quality gate in the process?
No it shows how important good prompting techniques are.
Nice work. I've also experimented with procedurally generated crossword puzzles, though I really wanted to constrain them to symmetric layouts like what you would find in the New York times which made it more difficult.
There's an outstanding issue and that is (from what I can tell) at least 75% of the answers correspond to relatively generic nouns or verbs.
Part of the deep satisfaction in solving a crossword puzzle is the specificity of the answer. It's far more gratifying to answer a question with something like "Hawking" then to answer with "scientist", or answering with "mandelbrot" versus "shape".
It might be worth going back and looking up a compendium of games released in the last couple decades, cross referencing them with their manuals, GameFaqs, etc. and peppering this information into the crossword.
Wanted to thank you again. I am currently working on an improved version providing more context. Your sentence "Part of the deep satisfaction in solving ..." made it into the prompt's rule-set. At this very moment I am only using the dataset of r/dota2 to make the testing easier and I look at the very first result with the new prompt:
Generated words and clues:
heroes: Characters with unique abilities in Dota 2, tasked with defeating the enemy's Ancient.
ragers: Players who overly react to in-game frustrations, often ruining the fun for everyone.
rage: A common emotion experienced by players sometimes leading to poor decision-making.
tachyons: Hypothetical particles that travel faster than light, having no place in an Ancient's mechanics.
healing: Essential support function often provided by certain heroes like Treant Protector.
burn: Refers to a mechanism used to deplete an opponent's mana, crucial in trilane strategies.
matters: In Dota 2, every decision, including hero picks, can significantly change the outcome.
fault: What a player will often blame when losing, rather than acknowledging their own mistakes.
support: Role in Dota 2 focused on helping the team, often with abilities to aid and sustain.
team: Group of players working together to win, where synergy and composition are key to victory.
Note that the Words themselves were not picked by OpenAI but rather a per-selection from the BERT Embeddings ML Algorithm but this time with more than just a word as context.
This is definitely going in the right direction. It's only sample size of 1 but i had to share it with you!
Absolutely! Keep me updated - these types of projects are definitely fun to explore. It'll also be interesting to explore alternative LLMs, as well as providing zero shot examples within the system prompt itself (if you haven't already).
I forgot to mention but it might also be worth exploring more classic NLP techniques like named entity recognition to score clues higher and lower in terms of overall specificity.
Have not done zero shot yet. At the moment i am experimenting with this https://arxiv.org/abs/2210.05245. Looks promising hoping to push an updated version soon but i am still fighting with it at the moment
Pretty reasonable. I don’t know where it pulled tachyons and it’s clue from though, that’s funny.
They talked about it in r/dota2 don't ask me why but they did :D
Thank you for your input. Your conceptual point on what makes crosswords fun is gold! Your practical advice is not only great but also very doable: Remember I used a 3.7million Reddit-Comments dataset which is a sub-optimal choice to generate words out of. (But i liked the headline behind it). So changing the dataset to something you mentioned (game compendiums, manuals, faqs) will probably increase the answers significantly.
In first puzzle I received, there was a clue for 1 across but there was no such place on the board. It was actually 2 across
~~~ UPDATE ~~~~
After a ~30 hours weekend coding marathon, I've just pushed a new version of the original joystick-jargon (r/gaming) and a new r/leagueoflegends puzzle live.
https://capsloq.de/crosswords/joystick-jargon
https://capsloq.de/crosswords/r/leagueoflegends
What changed?
- 5 new puzzles for r/gaming
- 6 new puzzles for r/leagueoflegends
- Old puzzles deleted
- New extraction algorithm (everything new: tokenizer, transformers, piplines, model, word and document embeddings, scoring, complete overhaul ...)
- New clue prompting
- Grid can now only contain diagonal black boxes (should guarantee intersections)
- Fixed numbering bug on the grid
- Did proof read each puzzle and some slight adjustments to guarantee puzzle integrity.
Warning: When i did proof read the League of Legends Q&A I noticed that I've never played that game so I couldn't verify everything!
Thank you very much to everyone who provided feedback to improve on v1.
I really hope you feel an increase in quality. I am looking forward for even more feedback and improving further.
Planning to use more suitable datasets in the future. It's super hard to get quality crossword list out of r/gaming.
Have fun puzzling! (please)
Echoing what vunderba said, it's hard for me to enjoy this because I'm used to the symmetric layouts of e.g. the NYT and the satisfying flavor of their clues.
However, this is well done and it inspired a thought- I wonder if it would be possible to procedurally generate word games, such as a mini crossword or word ladder or so on, as part of a language learning regime? Think Duolingo but for word puzzle fans.
As an example, you solve a mini crossword every day where 80% of the clues/answers are in English, and 20% are drawn from a progressive set of vocabulary in the other language.
TY for the feedback. I like the idea and its an interesting twist. Gamified learning is also a good market i think?
> Place in games where characters go to rest and replenish health or mana
> Bar
Very wrong, unless we’re simulating what someone unfamiliar with the gaming jargon would think - in which case very accurate.
Haha, true. I was sure it is "Inn" from Hearthstone Innkeeper but that's a case of LLM Hallucinations. See my other comment on how this can be improved but not solved completely imo
I had to reduce the size in my browser a couple or so times to see both the puzzle and all the clues at the same time. It might be better to have the clues in a separate scrolling region on the page.
The way the NYT does this on their web interface is nice. They have the puzzle in one column, the across clues in a second column, and the down clues in a third column. The clue columns each are scrollable.
It automatically scrolls to keep the clue for whatever word you have selected in view and highlights that clue, and also automatically scrolls to keep the clue for whatever word crosses that word at the particular square you have highlighted is also visible and marked in the margin of its clue list.
They do similar in their iPad app, but also below the puzzle show the clue for the selected word and for whatever word crosses it at the highlighted square. With that you can concentrate on the grid and a fixed clue area.
Thank you for that feedback. I agree NYT does this a lot better. I'll have to improve on that. Not gonna lie to you I suck at CSS/Styling/UX/UI will need to get some help regarding this. However i feel you and it's important for the fun of the game
Please respect your users. It’s frustrating to spend your time on a puzzle that might not even be plausibly solveable. If you’re going to ask strangers to do free testing you should at least bother to do some basic proofreading first.
Learnt my lesson. Truly felt (still) bad and sorry after people were saying they spent time and would not click solve easily. I only looked at it from a technical perspective and fully underestimated how serious the cross-wording community is. There was no bad intention and I will absolutely respect that from now on!
//EDIT. I've been working on it for approx. 10 hours today (still going) with the goal to put live a much better quality version. I feel the pressure and want this fixed myself. However it's quite a computational challenge i am giving my very best
The 13th puzzle is probably the first decent one. I still can't quite put together how it's getting the clues in some cases (BMing = using pedals? noly = Just this, no other?) but it's certainly the first one that's within reach of being reasonable.
TYVM for this feedback. I feel the exactly same way. This is purely a NON-PROOF-READ starting point, a field test of what AI can put out unsupervised. The prompt was changed in the latter puzzles those should be a tad better. But as already discussed there can be done a lot more to improve the quality of words and clues significantly.
I love crossword puzzles, and this is very well done! A while back I made a crossword puzzle generator using constraint programming. It is definitely way slower than whatever heuristic you are using, but it is very good at generating dense crossword puzzles using small word lists. This would be useful if you were interested in making a crossword puzzle using words related to a single game, for instance. You can read more on it here if you are interested: https://pedtsr.ca/2023/generating-crossword-grids-using-cons...
Thank you for sharing. I will check it out. And no worries regarding the speed. The website is cheating the 20 puzzles are pre-compiled so to say, the actually work was done in C++, the web server is just indexing an array of perfect puzzles at that point. TY!
Great effort!
Clickable link: https://capsloq.de/crosswords/joystick-jargon
thank you very much! And thanks again for the clickable link i didn't even notice
> 1: Allows movement between different areas in a game
> Unportal
what?
>19: An online forum section where gamers come together to discuss specific topics
> ITT
I don't think that's right...
Most of the questions just seem like normal crossword questions, but with the term "in games" added to it.
I'm not gonna sugarcoat it: this sucks. The crossword grids often have totally isolated words. 1 across and 2 down start with the same letter. The questions are nonsensical. I'd hardly go so far as to call it an interesting proof of concept.
Thank you for trying! Please see my reply on the other comment it fits here as well. This is a non proof read version and i agree with you consider this a starting point. Also feel free the try out the later puzzles (10+) i changed the prompt at some point because i noticed the same thing.
Edit: At some point I was unsure myself, if i don't know certain things about gaming or if the AI is making things up, haha, so this feedback is extremely valuable for me thank you
Tried it and think your approach is really cool.
Sadly, the clues and the words relating to them feel off, making the whole game rather unenjoyable.
TYVM for trying it out and feedback and it confirms what other users are saying. This is an unsupervised version with no human quality gate. Please consider this a starting point. That being said, I feel bad that you couldn't even enjoy it, that was not my goal. Sorry for that. I thought people would take a smile away regardless of some words/clues being bad. I have to admit i underestimated the appeal of quality. Some people say it gets at least a little bit better from puzzle #13, maybe you can get some enjoyment out of that? No worries if you are fed up at this point. Thanks for trying it out
>5. Grid Generation: Implemented a heuristic crossword algorithm to create grids and place words efficiently.
I always think about doing something similar for a similar project. Are you able to do it completely automatically or do you have to help finesse the words to fit?
The algo does allow for fully automatic crossword generation, let me try to summarize the general flow:
1. Begins with an empty grid and starts placing words horizontally from the top-left corner
2. For each word placement, it verifies that valid words can be formed vertically at each intersection point
3. It maintains a list of possible letters for each cell to ensure all constraints are satisfied
4. The generator consults a dictionary to find valid words that fit the curent grid state, allowing for diverse solutions
5. If no valid word can be placed, it may decide to insert a black square, carefully checking that doesn't violate any crossword rules
6. When it reaches an dead end, the system backtracks and tries different options
7. It employs smart heuristics to guide word selection, such as favoring longer words in certain positions
8. Throughout the process it automatically adjusts parameters like word length andblack square placement to find a valid solution
There is no manual intervetion, however the quality depends heavily on the input dictionary and tunable parameters.
Why is there no 1 across on the puzzle? There are clues for 1 across but not on the "board".
which puzzle number are you refering to?
I think i just saw it Puzzle #1 Clue #1. That's a bug in the numbering algorithm thanks for point it out.
It happens on puzzles 1, 2, 3, 6, 7, 11, 12, 13, 14, 15, 16, 17, 18, 19 and 20.
You right. Thank you for debugging that. And it's always the numbering 1. Definitely a bug in the algo. Will look into it
Awesome job! Also, might I suggest a bit of i18n? I can't read German..
Thank you! You right, no i18n yet, I can assure you that the URL https://capsloq.de/crosswords/joystick-jargon has the only playable crosswords at this moment. The other links in the Nav are not crossword related. Will definitely do i18n maybe even the crossword themselves in the future.
First once I loaded, 1 across was labeled 2
Unportal? What the fuck is unportal? I have played games for 30 years and I've never heard the term "unportal". Google gives no useful results. That clue made me angrier than any crossword clue I've ever seen.
TYVM for trying it out! Oh no, now I feel bad again. Please remember these are unsupervised generated clues by a Large Language Model (openai). More of a field test, however I now understand how serious you and others take crosswording. I vow to do better for the next crossword puzzles. It's not my intention to make people feel bad, sorry!
One more thing: Besides the obvious that clues and answers should make sense can you provide some insights in what makes a good crossword to you? I'd love to hear that from someone playing them for 30 years!
For me, a mixed crossword between all games doesn't make sense. You could give me 20 crosswords with Dota stuff in it and I would be able to solve 0, not even with hints it would be doable.
So maybe rather have some for League, some for CS, etc? Maybe you can do a mixed indie one with very popular games, or mixed Shooter. But then the questions have to be less difficult :D
TY for the feedback. I see what you are saying about the crossword getting too niche. For now this is mostly a technical decision since its easier to train on a more specific dataset. However i agree with you that the dataset should be expanded once the quality is there
Niche is fine, but mixing 3 niches is not.
Sorry I can't help, I've been playing video games for 30 years not crosswords :) I'd be keen the hear the same insights as you are.
Oh, I misread your first comment. Thought you were playing crosswords for 30 years that would've been a knowledge goldmine
AI hallucination
Agreed; probably derived from the 'portal' connections some games use to bridge areas of a map without having to consider their geographical layout.