DT1: Deon and Tong Pipelines RUs!

Here is the YouTube link for the video podcast:

YouTube player

The first Podcast and I find that I have no less than three podcast players in my WordPress, so apologies. The first is something called Podcast Player which seems broken. The second one is also called PodCast Player, but it seems like it provides a long list which is nice. I can’t seem to find a player on WordPress that points to the correct one, so sorry about that.

This is the WordPress Block which is the entire podcast with the latest first, it is the Podcast Player with a bunch of lines that come with JetPack called the Podcast Player and it needs the RSS Feed URL. And you can set it to just get a single episode but it’s not intuitive when in WordPress, the attributes are on the right and you have to click on Advanced to just get the single episode button like this. Otherwise it will just show the latest episode with a list.

DT1. Pipelines RUs! Paul, Deon & Tong

System design from the frontlines. When you are in a tiny startup, how do you structure things so that you have the maximum bang for the smallest buck? The easiest way is to try to linearize what you are doing so that you have a set of discrete pipeline parts that are fed by files. That way you turn a complex O(n^2) problem where complexity goes up by the square of the number of modules, to O(n) where adding one more module depends just on another component input. It's way easier to wire up. Plus if you have n developers, you minimize dependencies and can literally do things as fast as the slowest pipeline. Works in the real world too! And with larger companies. For you video watchers, apologies for the choppiness of the video, we are still learning how to make the chain work and will put in the show notes, but it looks like that default of creating a fabric that is 8Kx2K (way too big, but it covers all three screens) caused slow frame rates, but we've figure this out by shrinking the rates and using OBS > View > Stats to make sure the recording is working OK). Liner notes:  1. @deon's (yes that's a Mastodon address)'s first robot. https://slideplayer.com/slide/5672012/ where he made the PCB boards, and did all the programming by hand. Pretty amazing. 2. https://docs.google.com/presentation/d/18nxl0dtS_l0OgxKj0ZrGZlFyDzUVOLTXEpk3q89R3LI/edit#slide=id.gc6f59039d_0_0 for notes on system design and making it simple for startups, you can leave comments there. Or to @richtong 3. What all this then about the O(n^2) notation, so a quick chat about complexity theory. That is an estimate of how much harder computation gets. If you have 2 things and go to 100, if the algorithm is order -N or O(N), then the time to compute goes from 2 to 100 or 50x. But if it is O(N^2), then it goes from 4 to 10,000 so you can see as you add more things, life gets very complicated. Similarly, O(1) means constant in time, so when you go from 2 to 100 pipelines, if the time to compute stays at 1. That is a very nice thing. It means it is maximally parallel. Life is never really like that, but it's a good goal.https://en.wikipedia.org/wiki/Computational_complexity See https://tongfamily.com for details — Send in a voice message: https://podcasters.spotify.com/pod/show/richt/message

Show Notes

System design from the frontlines. When you are in a tiny startup, how do you structure things so that you have the maximum bang for the smallest buck? The easiest way is to try to linearize what you are doing so that you have a set of discrete pipeline parts that are fed by files. That way you turn a complex O(n^2) problem where complexity goes up by the square of the number of modules, to O(n) where adding one more module depends just on another component input. It’s way easier to wire up. Plus if you have n developers, you minimize dependencies and can literally do things as fast as the slowest pipeline. Works in the real world too! And with larger companies.

For you video watchers, apologies for the choppiness of the video, we are still learning how to make the chain work and will put in the show notes, but it looks like that default of creating a fabric that is 8Kx2K (way too big, but it covers all three screens) caused slow frame rates, but we’ve figure this out by shrinking the rates and using OBS > View > Stats to make sure the recording is working OK).

Liner notes: 

  1. @deon‘s (yes that’s a Mastodon address)’s first robot where he made the PCB boards, and did all the programming by hand. Pretty amazing.
  2. Google Slides for notes on system design and making it simple for startups, you can leave comments there. Or to @richtong@mastodon.social
  3. What all this then about the O(n^2) notation, so a quick chat about complexity theory. That is an estimate of how much harder computation gets. If you have 2 things and go to 100, if the algorithm is order-N or O(N), then the time to compute goes from 2 to 100 or 50x. But if it is O(N^2), then it goes from 4 to 10,000 so you can see as you add more things, life gets very complicated. Similarly, O(1) means constant in time, so when you go from 2 to 100 pipelines, if the time to compute stays at 1. That is a very nice thing. It means it is maximally parallel. Life is never really like that, but it’s a good goal. https://en.wikipedia.org/wiki/Computational_complexity

See https://tongfamily.com for details

I’m Rich & Co.

Welcome to Tongfamily, our cozy corner of the internet dedicated to all things technology and interesting. Here, we invite you to join us on a journey of tips, tricks, and traps. Let’s get geeky!

Let’s connect