Gorilla Newsletter 70

Welcome back everyone 👋 and a heartfelt thank you to all new subscribers that joined in the past week!

This is the 70th issue of the Gorilla Newsletter - a weekly online publication that sums up everything noteworthy from the past week in generative art, creative coding, web3, tech and AI - with a spritz of Gorilla updates.

If it's your first time here, we've also got a discord server now, where we nerd out about all sorts of genart and tech things - if you want to connect with other readers of the newsletter, come and say hi: here's an invite link!

That said, hope that you're all having an awesome start into the new week! Here's your weekly roundup 👇

All the Generative Things

Beyond Wave Function Collapse + EPC 2024

EPC 2024 (Everything Procedural Conference) took place earlier this year in April, and over the weekend I discovered that the talks given at the conference had been published in their entirety a while ago already - unsurprisingly they turn out to be a treasure trove for all sorts of different cutting edge generative and procedural techniques.

You can find a run-down and descriptions for them over on the official website, as well as and the complete archive of recordings (also including talks from past years) over on YouTube:

For generative art purposes, one talk in particular stands out this year, it's by Paul Merrell who you might recognize as the brain behind the original series of papers that introduced the Model Synthesis algorithm for procedural pattern generation, a technique that we more commonly know today as Wave Function Collapse in genart and game dev circles. While Merrell had a hunch early on that his method would find great application in video games, WFC only gained traction after Maxim Gumin published a Github repository that provides a tangible implementation of the algorithm:

In a nutshell, WFC is essentially a procedural content generation algorithm that's primarily used to create textures, patterns, and game levels (assembling tilemaps) in a way that looks coherent and follows certain rules or constraints. The algorithm operates by determining the state of each cell in a grid (like a tile or pixel, but can also be extended to irregular grids) based on constraints derived from example input patterns. It "collapses" possibilities for each cell into a single outcome that aligns with its neighbors, ensuring the generated output matches desired properties - for a digestible introduction you should check out Coding Challenge #171 in which Daniel Shiffman tackles a full implementation of the algo:

Famous instances of the algorithm in action, out in the wild, are the iconic Townscaper and the lesser known mobile game Bad North, both developed by Oskar Stålberg. It's important to note here that Stålberg additionally extended the method by applying it to an irregular grid (I wrote about a bit about the construction of such grids in issue #59). A more recent example is Tiny Glade that I believe to also employ wave function collapse for adjoining textures. In this manner WFC is arguably a revolutionary technique for video game aesthetics. In generative art there's also a very cool recent example for it, loackme's and Karim Safa's Nullmachines also leverages WFC to create the animated endlessly looping conveyor belts:

A limitation of WFC however is that it's tile based and confined to a grid - and as the title of the Paul's conference talk reveals it revolves around a new method to achieve procedural patterns without relying on tiles at all. Having worked on this for over 2 decades since the publishing of the Model Synthesis paper, Paul introduces us to Graph Grammars:

I highly recommend watching the talk, while quite technical, the notions can be understood quite easily. Merrell explains that there are actually already many procedural techniques that rely on shape grammars—rules that dictate how certain shapes can combine with each other into new shapes. The caveat of these previous methods is that these combinatory rulesets are generally manually designed by experts. In contrast, Merrell's proposed method generates the shape grammar automatically, based off of an example texture and/or sample shape - and that without relying on a grid of tiles for the assembly of new patterns, but rather modeling the result as a graph. I'll leave it at that though, if you only have time for a single video this week, this is the one!

Markov Junior: A Probabilistic Programming Language

By chance, I discovered another important Github repo from Maxime Gumin - not by going through his GitHub profile, but rather through a post from the @algoritmic TwiX account that showcased intriguing procedurally generated voxel towers, linking back a repo from Gumin titled Markov Junior:

To put it simply, Markov Junior is a sort of esoteric programming language that draws inspiration from Markov Algorithms, and is in essence a general purpose tool geared towards procedural content generation. Not to be confused with Markov Chains, probabilistic mathematical models used to describe state transition systems - Markov algorithms on the other hand are abstract symbol manipulation and transformation machines.

An algorithm in this setting is comprised of two things: an alphabet of symbols, accompanied by a set of rules or instructions to perform string transformations and rewrites on the symbols in the alphabet - more formally this is described as a scheme; a finite ordered set of substitution formulas. This sounds more complicated than it actually is, to give an example, a rule could simply be "replace the letter a with the letter b" - Wikipedia provides a more sophisticated example:

From the Wikipedia page on Markov Algorithms

The github repo itself provides a brief explanation of Markov Algorithms, and also showcases a more practical example, converting the binary representation of a number to its unary equivalent:

With Markov Junior these rewrite protocols gain application in procedural content generation - in the 2D and 3D settings programs essentially describe types of cellular automata, some of which might look very familiar at first glance, like maze generators for instance. The distinction is that these procedural patterns become very easy to define with just a couple of substitution rules:

On Collision Detection Algorithms by Lean Rada

Lean Rada wrote, not one, but two articles on collision detection; the first article talks us through how sorting objects can go a long way for the purpose of reducing the number of collision checks that need to be computed, all while introducing us to the elegant and efficient Sweep-and-Prune algorithm - followed up by a second installment where Lean fills in the missing parts and explains a more complete implementation of the algorithm:

The common issue with naive approaches to collision detection is the number of checks required to determine if any two shapes in the simulated world intersect or not—in the worst case, each shape needs to be checked against every other shape, leading to a computationally expensive O(n^2) problem.

The Sweep-and-Prune algorithm is one of the methods that reduces the number of necessary checks. In essence, this is done by sorting objects based on their positions along a chosen axis and creating intervals that represent the space each object's axis-aligned bounding box (AABB) occupies along that axis. This allows the algorithm to focus only on checking collisions between objects whose intervals overlap, thus quickly narrowing down the number of object pairs that might be colliding. The algorithm then tracks the minimum and maximum extents of each object’s AABB as separate points in a sorted list, and then sweeps a line across this list. As the sweep progresses, it keeps track of which objects' intervals are currently active (overlapping with the sweep line) and checks for potential collisions only between those objects, efficiently identifying pairs of shapes that could potentially intersect. Then we can fire-up a more accurate collision check for the shape we're dealing with to determine if there is in fact a collision.

Now I'm oversimplifying this with my explanation; make sure to check out both of Lean's articles, they're wonderfully written and accompanied by more than a few nice interactive snippets to exemplify the ideas throughout - the latter part of the second article discusses further optimizations that we can apply to further speed up the overall procedure:

While I've implemented spatial hashing (grid/quadtree lookups basically) on several occasions, I've never actually used Sweep-and-Prune - what's interesting is that the two can actually be combined, for instance you could do a sweep check on each grid cell to potentially further reduce the number of collision checks that need to be made. I'm not entirely certain if it would lead to that much more of a performance gain however.

On Visualizing Sound - Are.na Editorial

Be it Spectrograms produced by passing a sonic signal through the Fourier Transform, or the geometric compositions that visualize different chords and melodies on an instrument - representing sound in the visual domain via graphical notation, as a form of translation between the senses, has long been a fertile ground for creative expression. A recent Are.na editorial piece highlights this relationship through Travess Smalley and Daniel Lefcourt's work with graphic notation, offering a compelling look at its contemporary relevance, and how it lends itself as a perfect playground for building interpretational systems:

Both artists agree that the applications of graphic notation extends beyond that of mere artistic expression, and can serve as a powerful educational tool for teaching abstraction, as a rule-based approach to creativity; rather than relying solely on intuitive or subjective decision-making, students have to derive form and composition from a systematic computational process that can translate between the sonic and visual domain. Make sure to check out Travess Smalley's Visualizing Sound: Tools, Notations, Drawings, Graphics, and Paintings channel and Daniel Lefcourt's Graphic Traces channel towards that end, they're chock-full of creative music and sound visualizations examples, some of which are talked about in more detail throughout the article itself.

If you're not familiar with Are.na, I briefly wrote about it in issue #47 and #48 of the Newsletter, but in essence it is a social networking platform designed for collaborative curation, organization, and sharing of content. It was created to provide a space for thoughtful and creative individuals to collect and organize information in a more structured and intentional way than typical social media platforms.

Other Cool Generative Things

Chris Ried is back with another issue of his fantastic newsletter, he seemingly always manages to find interesting things that I miss - go check it out and give it a ried.
It seems that there is no end to Nolen Royalty's creative experiments, less than 48 hours after giving a talk about his 1 million checkbox website, he gives second talk, at another conference, about a different project of his. If you're curious about programming flappy bird in the MacOS finder, you should catch up with what Nolen's been up to.
And for our weekly installment of mini code shenanigans, I want to highlight a post from Michael Rostoki, whom I am a big fan of ever since discovering his stuff during Genuary - he frequently shares short tweet sized Lua sorceries for the beloved Pico 8 fantasy console.
To not reinvent the wheel, the P5 sound library is being revisited following the 2023 sound fellowship, such that it'll act as a thin wrapper around tone.js - which is already designed with creative coding in mind. You can read more about this endeavour here.

Tech and Web Dev

Font with Built in Syntax Highlighting

If you've ever started your own programming blog, you probably found yourself in a situation where you want to showcase some code snippets on your page, and actually have them highlighted as proper syntax, and not just as plain text. And in that scenario there's no way around using some third party library that takes care of code and syntax highlighting. This blog uses the popular highlight.js as a solution towards that end.

While these libraries are super handy, they also come with some downsides. For instance, they add an additional overhead to the pageload, either because they're additional resources that need to be loaded in from an external place, or by making your page's size more bloated by being directly included. That's also beside the code itself running to modify the DOM and color the syntax accordingly - and in these modern times we care a lot about making our pages blisteringly fast to load.

A recent article from the Glyph Drawing Club Blog, explores an intriguing approach to syntax highlighting without relying on JavaScript at all — but rather by building syntax highlighting directly into the font itself, and this through certain OpenType font features. The method uses colored glyphs and contextual alternates to identify and highlight different elements of code syntax:

This approach obviously comes with its own set of pros and cons. The most impressive thing about it is how simple it makes code highlighting - you just have to specify the modified font via the font-family css rule and apply it to the code html tag. An added benefit is that it also make syntax highlighting possible in the textarea and input tags that were previously impossible to style in that manner. On the flipside, the limited pattern-matching capabilities compared to regex-based solutions may also result in less accurate syntax highlighting in complex scenarios — although... the Hacker News comment section argues otherwise, bringing up that it might just be possible to implement that:

Another minus is that it requires one to modify the font file itself, which is not a very straightforward thing to do, firstly because specific software is needed for that, and secondly because many font licenses simply don't actually allow for modification. From a broader perspective however, it's a really cool project - while it may not replace traditional syntax highlighting libraries just yet, it's an interesting alternative to keep in mind for scenarios where you need something simple and performant, without caring too much about extensive customization.

CSS Day 2024 Talks + Sara Soueidan CSS Accessibility

The talks given at CSS Day 2024, held earlier this year in July, have been gradually published over the past couple of weeks on the Web Conference Amsterdam YouTube channel - all of them are gems and worth queuing into your to-watch list. One talk that I want to highlight in particular, is by none other than Sara Soueidan, who's arguably become a CSS superstar at this point given her contributions over the years. In her talk she tackles CSS from an accessibility perspective, and how certain rules can have a severe impact on the accessibility of the elements in the user interface that they style, all while providing remedies and alternatives to those faux-pas:

CSS does not only affect the visual presentation of elements, but also indirectly impacts the information that is exposed by elements in the accessibility tree, and whether it's exposed at all. For instance, lists are crucial for screen readers because they convey important structural information about grouped items, and are often used to indicate sequences of related interface elements, like navigation links. For visual purposes, however, developers might use list-type: none to remove visual list markers, particularly for navigation elements - this also inadvertently strips them of their semantic accessibility labels in certain browser/screen reader pairings. A better solution here is to use list-style-type: "". Sara points out an interesting article on that matter here.

Another example is relying on the popular display: none CSS hack as a solution to visually hide elements from a webpage, not only does it achieve that, but also completely removes it from the accessibility tree - no bueno - using a utility class to this end, for visually hiding content, while keeping it accessible is a better approach:

But that's just scratching the tip of the iceberg though, she covers many more considerations, referencing many more interesting resources, and is overall a must-watch if you want to get better at designing for accessibility. Bottom-line is that you should always fire-up a screen reader and test rigorously.

The Process behind Redesigning Piccalilli

And to cap off the tech section this week, I'd like to highlight an article from one of my favorite tech blogs, Picalilli. More precisely, it's a publication that is run by Set Studio, with Andy Bell acting as the primary writer. Picalilli's made a couple of appearances on the newsletter in the past, in issue 57 where they pitch in on the CSS Masonry Grid debacle (that is still an ongoing debate at this point), as well as in issue 24 with a follow up to their iconic CSS reset article, A more Modern CSS Reset.

This week they make a return with a very special article, the third and final in a series of three, documenting the redesign of the blog, from the ground up to finishing touches, in an ongoing effort to become a more frequent and serious online publication overall:

Link to First Article | Link to Second Article

And this three part series is just absolute gold, since it documents the entire process it's especially insightful for my own purposes as I'm still working on the redesign of this very blog you are currently reading (I promise it'll be ready at some point), and because I've never done something like it before I'm trying to actively collect each and every piece of advice I can come across. In particular when it comes from a web design firm. I found that I'm actually doing some things the right way, such as mood boarding to find potential design direction of the project, as well as using Figma to prototype, etc. But equivalently, there's also a lot of things that I've missed. Anyway, big must-read!

Other Cool Tech Things

Stefan Judis covers a cool and mind-boggling clipboard copy-paste magic trick that was originally shared by Cyrus Roshan in a post back from 2022. Stefan explains how it works in detail, and I might just turn this one into a proper section in the next newsletter.
Another update from the Ghost team on their efforts of integrating the CMS with the activity pub protocol - and they've made some big strides since they got started a couple of months ago. I've genuinely been eagerly awaiting their newsletter every Sunday.

The Advent of AI Powered Code Editors

The AI powered code editor Cursor has been making a buzz over on social media in the past weeks: it's a direct fork of VS Code that extends its capabilities with AI features, directly built into the editor itself. This essentially replaces the need for running Github copilot separately, as code completion and edits are now provided out of the box, in addition to a contextual chat that sees your codebase and lets you directly apply the LLM generated suggestions to your code:

It does come at a cost though, but lets you try it out for 2 weeks free of charge, so it might be worth checking out. It also runs all the existing VS Code extensions that can be found on the marketplace, so you're really not missing out on anything from the regular VS Code. I have yet to try it out, but it's on my to-do list this week, I have a hunch however that it'll probably be a no-brainer since I already use LLMs on a daily basis for all sorts of tasks - recently I've been especially impressed by Claude Sonnet 3.5 for programming purposes.

But I'm not alone with that, unarguably many programmers already use LLM's for programming purposes - the StackOverflow Developer Survey that came out earlier this year shows that 61% of programmers are already using AI in their development process, so it just makes sense to directly incorporate AI within the code editor and perfectly fill out this latent market gap:

Link to the AI section of the StackOverflow Developer Survey

Cursor's already been labeled as the "VS Code Killer" - and that might just be the case considering that it's raised 60$ million in funding in the first 2 months after launching, with the majority of funding coming directly from OpenAI. Regardless of that however, I can definitely see that this will be the new status quo when it comes to the code editors of the future.

The influential Andrej Karpathy voices his thoughts on that matter with a Tweet of his, basically stating that he can't imagine going back to programming without AI powered assistance - it does however also raise concerns about the future generations of programmers that don't have to go through learning how to code "the hard way":

**I basically can't imagine going back to "unassisted" coding at this point** | Link to Thread

Music for Coding

This week I've got an album with historic significance for you; Sacbé is Mexico's very first fusion band, and their titular album is nothing short of a masterpiece that holds very strong even in 2024 - I found an interview with Enrique Toussaint that recounts the initial days of the band, and how they arrived at their very tight and dynamic sound, a perfect read while listening to the album:

And that's it from me—hope you've enjoyed this week's curated assortment of genart and tech shenanigans!

Now that you find yourself at the end of the Newsletter, you might as well share it with some of your friends - word of mouth is till one of the best ways to support me! Otherwise come and say hi over on my socials - and since we've got also got a discord now, let me shamelessly plug it again here. Come join and say hi!

If you've read this far, thanks a million! If you're still hungry for more generative art things, you can check out last week's issue of the newsletter here:

And a backlog of all previous issues can be found here:

Cheers, happy coding, and again, hope that you have a fantastic week! See you in the next one!

- Gorilla Sun 🌸