I modded Minecraft into a godlike music visualizer (+Distant Horizons)

I'm taking minecraft music sync to the next level. My dream of turning minecraft into the ultimate music visualizer has finally been fully realized. I now have the power to execute ANY combination of minecraft commands with frame perfect precision locked in to the timeline of any song. The possibilities are endless!

How did I do this?
This music visualizer is an amalgamation of custom tools I've made, as well as incredible mods made by others. Huge shoutout to ‪@Distant_Horizons‬ for their incredible work. In my previous video where I wanted a 256 chunk render distance, I was subjecting my poor computer to loading every single chunk. Distant Horizons LOD files made this video render 10x faster at least.

1. Built Unity Integration
- I built a Unity Project which mirrors the Minecraft player cam to a Unity camera. This allows me to keyframe the camera motion using all the nice animation features the Unity editor has to offer.

2. Built Minecraft Mod
- To connect Minecraft to my Unity scene, I built a Minecraft mod which contains a socket, allowing for bidirectional communication between Unity and MC.

3. Edit Shader Timing
- To maintain the illusion that Minecraft is running at normal speed while we're recording, I need to make sure that the in game time advances in lockstep with the frame rate. With some simple math, we can take 60fps video, and 20 MC ticks/s, and use the very convenient /tick step command every 3 frames to maintain correct timing in game. This works great for the game time, however, shaders operate off of a different timing system. The shader I wanted to use (BLISS), didn't have world time animation as an option, so I learned how to tweak it myself. All I had to do was convert parts of the shader that used the "frameTimeCounter" uniform to use the "worldTime" uniform.

The main downside of this setup is that it is not very user friendly yet. However, if there's interesting, with enough support on Patreon I would be able to dedicate my time to making a public version.

Patreon:   / 95792111  
Discord:   / discord  
Twitter:   / doodlechaos  
www.doodlechaos.com

Mods/Shaders:
https://www.curseforge.com/minecraft/...
DistantHorizons-2.1.0-a-1.20.6-noForge
dreamvis-1.0.0 (my mod, not public)
fabric-api-0.100.0+1.20.6
indium-1.0.30+mc1.20.4
iris-1.7.0+mc1.20.6
sodium-fabric-0.5.8+mc1.20.6
worldedit-mod-7.3.2

Music:
   • Julius Dreisig & Zeus X Crona - Invis...