parallel universe movie trailers

This is more output from my (heavily modified) version of riversHaveWing's VQGAN+CLIP(z-quantize method).ipynb code. (Link to her code below.)

[Ok, YouTube compression made it even less watchable than it should be. I'll try improve that with some... creativity? perhaps pre-filtering before outputting with an average between the previous and current frame inserted, and a higher frame rate. We'll see how that looks next time.]

The idea is you get a new set of (vaguely) movie-plot inspired prompts generated every 10 seconds, then another, and another...

This is done by modulating the optimizer's learning rate. It has a burst of 'stupidly high' learning for the first frame of a sequence, making effectively jump off in a random direction in z-space. (It will overshoot whatever it thought it was aiming at and end in the foliage around the edges... try multiply regular z values by 10 and you'll see what I mean.)

Then it ramps up for a while, and ramps back down, settling on a low-ish value. The reason it's so cartoon-like is because I forgot to clear an input, and didn't notice for a few hours. It will not not happen again.

There is an index which links the timestamp to the content it used to create that 10ish second sequence at:
https://github.com/zenith-parsec/Trai...

This i

(There are a couple of swears in the page, as it's automatically generated by an AI which is able to choose "appropriate" concepts, and my dictionaries are general. So don't be all shocked. And don't visit it you will be.)

This is the code I based my tool on.
https://colab.research.google.com/dri...

I added a couple of different movie plot generators, one base don trivial templates and the other based on two EBNF representations of English. The first was based on the answer to a stackoverflow question:

https://english.stackexchange.com/que...

And the other basically tried to 'reverse the anti-grammar" used by the WordNet sentence templates, into EBNFish templates with some 'find/replace' logic added to fix up some context things.
E.g. verbs like "S add s X to Y" can work.
The 'EBNF' (where E means extended, and yeah, this is non-standard ) for that looks like:

STATEMENT_11: MAGIC_STATEMENT_11|ORIG_STATEMENT_11
QUESTION_11: DO OBJ_SUBJECT ~verb#11~ OBJ_OBJECT
ORIG_STATEMENT_11: OBJ_SUBJECT ~verbs#11~ OBJ_OBJECT
MAGIC_STATEMENT_11: {~verbs#11~,/"/OSQ /,/"/" OT /,/_/" OBJ_OBJECT "/,/OSQ /OBJ_SUBJECT "/,/ OT / OBJ_TARGET/}
ALT_QUESTION_11: W_DO_O OBJ_SUBJECT ~verb#11~ OBJ_OBJECT

ORIG_STATEMENT_11 is based on frame sentence 11 from WordNet 3.x.

Unfortunately some of these were broken by other changes during this run... so you get what you get. These are used in the 'non-template' text.

(Side note: it's amazing just how bad some of the wordlists for, e.g. 'verb,verbs, verbed, verbing' are. Also, most grammars are for parsing, not generation. At least a few rules still need to have more stuff added to them, just to be syntactically ok. ;/ (and I know the right words for stemming/conjugation/word forms/lemmas etc.. trying to keep this understandable. thanks if you were going to comment.)