Reading list July 2023

The list of things I read or watched this month The right way to define a C function with no arguments - https://www.youtube.com/watch?v=VsRs0H4hXEE TREE-STRUCTURED CONCURRENCY — 2023-07-01 - https://blog.yoshuawuyts.com/tree-structured-concurrency/ Myths Programmers Believe about CPU Caches - https://software.rajivprab.com/2018/04/29/myths-programmers-believe-about-cpu-caches/ Eventual vs Strong Consistency in Distributed Databases - https://hackernoon.com/eventual-vs-strong-consistency-in-distributed-databases-282fdad37cf7 Accounting For Developers, Part I - https://www.moderntreasury.com/journal/accounting-for-developers-part-i Accounting For Developers, Part II - https://www.moderntreasury.com/journal/accounting-for-developers-part-ii Accounting for Developers Part III: Building a Lending Marketplace - https://www.moderntreasury.com/journal/accounting-for-developers-part-iii What is a Wire Transfer? - https://www.moderntreasury.com/learn/what-is-a-wire-transfer What is RTP? - https://www.moderntreasury.com/learn/what-is-rtp vTables for Game Hacking & VMT Hooking - https://www.youtube.com/watch?v=HfrBdf-hM28 API vs. Messaging — How to Choose Which One to Use? - https://betterprogramming.pub/api-vs-messaging-how-to-choose-which-one-to-use-d6634599d2bd Kubernetes Basics: Pods, Nodes, Containers, Deployments & Clusters - https://www.youtube.com/watch?v=6IPu3WU_M0o Are You Sure You Want to Use MMAP in Your Database Management System? - https://db.cs.cmu.edu/papers/2022/cidr2022-p13-crotty.pdf Content-based filtering & collaborative filtering (Building recommendation systems with TensorFlow) - https://www.youtube.com/watch?v=v90un9ALRzw What are Business Rules? It’s not this. - https://www.youtube.com/watch?v=FbYcIqVmGRk Demystifying the Unusual Evolution of the Netflix API Architecture - https://www.youtube.com/watch?v=Uu32ggF-DWg A Gentle Introduction to Linear Algebra - https://machinelearningmastery.com/gentle-introduction-linear-algebra/ Introduction to Amazon Personalize - https://www.youtube.com/watch?v=3gJmhoLaLIo Understanding your Data with Amazon Personalize - https://www.youtube.com/watch?v=TEioktJD1GE Solving Real World Use Cases with Amazon Personalize - https://www.youtube.com/watch?v=9N7s_dVVWBE how do computers create code? - https://www.youtube.com/watch?v=2y1IgW2T8bo What if writing tests was a joyful experience - https://blog.janestreet.com/the-joy-of-expect-tests/ Quickcheck for Core - https://blog.janestreet.com/quickcheck-for-core/ Basics of Mathematical Notation for Machine Learning - https://machinelearningmastery.com/basics-mathematical-notation-machine-learning/ A Real Use Case of Goroutines With Package Context In Golang - https://www.youtube.com/watch?v=5QyxqhLnKmM Engineering Intensity - https://ruiper.es/posts/engineering_intensity/ Implementing a Transactional Outbox Pattern with DynamoDB Streams to Avoid 2-phase Commits - https://medium.com/ssense-tech/implementing-a-transactional-outbox-pattern-with-dynamodb-streams-to-avoid-2-phase-commits-ed0f91e69e9 StatQuest: K-nearest neighbors, Clearly Explained - https://www.youtube.com/watch?v=HVXime0nQeI BRIDGING FUZZING AND PROPERTY TESTING — 2023-07-10 - https://blog.yoshuawuyts.com/bridging-fuzzing-and-property-testing/ 289 - CONFIABILIDADE & ESCALABILIDADE em aplicações com uso intensivo de DADOS | theWiseDev NFR - https://www.youtube.com/watch?v=EXwJwwOJ1UY Why ScyllaDB is Moving to a New Replication Algorithm: Tablets - https://www.scylladb.com/2023/07/10/why-scylladb-is-moving-to-a-new-replication-algorithm-tablets/ A problem so hard even Google relies on Random Chance - https://www.youtube.com/watch?v=lJYufx0bfpw Are You Sure You Want to Use MMAP in Your Database Management System? - https://www.youtube.com/watch?v=1BRGU_AS25c Digital Wallet - https://www.moderntreasury.com/learn/digital-wallet What is ACH? - https://www.moderntreasury.com/learn/what-is-ach How simple can a programming language be? - https://www.youtube.com/watch?v=Fg5En6pbsDA Kubernetes Deployment vs. StatefulSet vs. DaemonSet - https://www.youtube.com/watch?v=30KAInyvY_o How to Optimize Caching for Transactions - https://audreyccheng.com/blog/transaction-caching/ How to use mongoDB change streams as a powerful event-driven engine - https://dev.to/woovi/how-to-use-mongodb-change-streams-as-a-powerful-event-driven-engine-4d9c Making WebAssembly even faster: Firefox’s new streaming and tiering compiler - https://hacks.mozilla.org/2018/01/making-webassembly-even-faster-firefoxs-new-streaming-and-tiering-compiler/ Life of a Packet [I] - Michael Rubin, Google - https://www.youtube.com/watch?v=0Omvgd7Hg1I Data Lakehouse Architecture & Use-Cases - https://www.youtube.com/watch?v=hB6olelYhr0 Making FinTech systems safe, reliable, and scalable with Redpanda - https://redpanda.com/blog/data-streaming-for-financial-services Announcing Rust 1.71.0 - https://blog.rust-lang.org/2023/07/13/Rust-1.71.0.html Focusing on “Entities” leads nowhere good. - https://www.youtube.com/watch?v=v5Fss4fCl8c How Functions Function - https://www.youtube.com/watch?v=SqT5YglW3qU Single vs. Double Entry Accounting - https://www.moderntreasury.com/learn/single-vs-double-entry-accounting Bank Reconciliation - https://www.moderntreasury.com/learn/bank-reconciliation What is a Ledger? - https://www.moderntreasury.com/learn/what-is-a-ledger How Big Is A Character? - https://www.youtube.com/watch?v=BAGcSyrWiPs Scylla Summit 2022: The Future of Consensus in ScyllaDB 5.0 and Beyond - https://www.youtube.com/watch?v=LoriHUqrzso I Cheated, and Gave Myself Infinite Hearts in Aladdin for SNES (snes rom hacks) - https://www.youtube.com/watch?v=2ZnAS5_0Z3I Accounting for Computer Scientists - https://martin.kleppmann.com/2011/03/07/accounting-for-computer-scientists.html Andrew Kelley - Practical DOD - https://vimeo.com/649009599 Anicet Nougaret - Insights from making a DNN & CNN library from scratch in Rust - https://www.youtube.com/watch?v=GjUM8ejy6kE What is Database Sharding? - https://www.youtube.com/watch?v=XP98YCr-iXQ Handles are the better pointers - https://floooh.github.io/2018/06/17/handles-vs-pointers.html Getting Hands on with io_uring using Go - https://developers.mattermost.com/blog/hands-on-iouring-go/ Implementing Raft: Part 0 - Introduction - https://eli.thegreenplace.net/2020/implementing-raft-part-0-introduction/ Implementing Raft: Part 1 - Elections - https://eli.thegreenplace.net/2020/implementing-raft-part-1-elections/ Implementing Raft: Part 2 - Commands and Log Replication - https://eli.thegreenplace.net/2020/implementing-raft-part-2-commands-and-log-replication/ Implementing Raft: Part 3 - Persistence and Optimizations - https://eli.thegreenplace.net/2020/implementing-raft-part-3-persistence-and-optimizations/ How do games like Prince of Persia manipulate time? | Bitwise - https://www.youtube.com/watch?v=GYLBCTTyHsI What’s new with io_uring - https://kernel.dk/io_uring-whatsnew.pdf Hash chain - https://en.wikipedia.org/wiki/Hash_chain Things I Wish I Knew Before Building a Ledger - https://www.andriosrobert.com/p/things-i-wish-i-knew-before-building Back to Basics: commands, events, and messages - https://milestone.topics.it/2023/05/25/back-to-basics-messages.html The looming demise of the 10x developer: Why an era of enthusiast programmers is coming to an end - https://blog.testdouble.com/posts/2023-07-12-the-looming-demise-of-the-10x-developer/ NumPy vs SciPy - https://www.youtube.com/watch?v=l3s-_8uTBVA The basics of the InnoDB undo logging and history system - https://blog.jcole.us/innodb/ Visualizing the impact of ordered vs. random index insertion in InnoDB - https://blog.jcole.us/2014/10/02/visualizing-the-impact-of-ordered-vs-random-index-insertion-in-innodb/ Coroutines for Go - https://research.swtch.com/coro 290 - SQL ou NoSQL: EIS A QUESTÃO! 🤔 | theWiseDev SQL - https://www.youtube.com/watch?v=mP11yY6uKmg Step Function Callbacks with AWS Lambda | Console Tutorial - https://www.youtube.com/watch?v=Uacjxu_AJTI Multi-tenant data isolation with PostgreSQL Row Level Security - https://aws.amazon.com/blogs/database/multi-tenant-data-isolation-with-postgresql-row-level-security/ MySQL Row-Level Security - https://satoricyber.com/mysql-security/mysql-row-level-security/ Simulation Testing For Liveness - https://tigerbeetle.com/blog/2023-07-06-simulation-testing-for-liveness/ Basics of Linear Algebra for Machine Learning - https://machinelearningmastery.com/linear_algebra_for_machine_learning/ Binary Comparisons for Patch Diffing - BinDiff Tutorial - https://www.youtube.com/watch?v=n06QSoICU6c How real user monitoring works in Grafana Cloud Frontend Observability - https://www.youtube.com/watch?v=5MfpcwTEQ6g Observe your Rust application with Quickwit, Jaeger and Grafana - https://quickwit.io/blog/observing-rust-app-with-quickwit-jaeger-grafana DevOps vs SRE vs Platform Engineering | Clear Big Misconceptions - https://www.youtube.com/watch?v=an8SrFtJBdM DDD is just giving a $h!t about your Domain - https://www.youtube.com/watch?v=i0aGAdgbG7A Bernard Kolobara – Lunatic - Erlang-inspired runtime for WebAssembly - Rust Vienna May 2023 - https://www.youtube.com/watch?v=VhDUGLdj5hY Differences between Apache Kafka and Redpanda: Bypass page cache & optimize for direct memory access - https://www.youtube.com/watch?v=ugZkzonAaqs Vertical Slice Architecture, not Layers! - https://www.youtube.com/watch?v=L2Wnq0ChAIA Sending generated traces to Grafana Tempo - https://www.youtube.com/watch?v=V5GtJRZtZ90 A Good Rust Stack - https://www.youtube.com/watch?v=pocWrUj68tU Make invalid states unrepresentable - https://geeklaunch.io/blog/make-invalid-states-unrepresentable/ Book Review: Nine Lies About Work - https://www.daviddaly.me/2023/07/book-review-nine-lies-about-work.html Building for Failure - https://encore.dev/blog/building-for-failure Why fsync(): Losing unsynced data on a single node leads to global data loss - https://redpanda.com/blog/why-fsync-is-needed-for-data-safety-in-kafka-or-non-byzantine-protocols?utm_source=twitter&utm_medium=social&utm_campaign=may16_social&utm_assettype=blog&utm_assetname=fsync The big picture: How Google Photos scaled rapidly on Spanner - https://cloud.google.com/blog/products/databases/google-photos-builds-user-experience-on-spanner/ Full Circle - https://taliaringer.wordpress.com/2023/07/22/full-circle/ 6 SQL Joins you MUST know! (Animated + Practice) - https://www.youtube.com/watch?v=Yh4CrPHVBdE How This Expression Caused a Global Internet Outage - youtube.com/watch?v=DDe-S3uef2w Square vs. Stripe - https://www.investopedia.com/articles/personal-finance/021715/square-vs-stripe.asp Qual a diferença entre Instituição de Pagamentos e Instituição Financeira?​ - https://www.iugu.com/blog/instituicao-de-pagamentos O que são arranjos de pagamentos e para que eles servem? - https://www.iugu.com/blog/o-que-sao-arranjos-de-pagamentos Entenda o que é CIP e veja as mudanças para marketplaces - https://www.iugu.com/blog/cip-camara-interbancaria-pagamentos-o-que-e Building an an Early Stage Startup: Lessons from Akita Software - https://blog.pragmaticengineer.com/building-an-an-early-stage-startup/ O que é uma instituição de pagamento regulamentada pelo BACEN? - https://www.iugu.com/blog/instituicao-de-pagamento-regulamentada-bacen Como funciona o mercado de Marketplace no Brasil? - https://www.iugu.com/blog/mercado-de-marketplace-no-brasil Token triplo: o que é e como protege os pagamentos digitais? - https://www.iugu.com/blog/token-triplo Instituições de pagamento - https://www.bcb.gov.br/estabilidadefinanceira/instituicaopagamento Arranjo de Pagamentos: O que sua Startup precisa saber sobre esse sistema - https://ndmadvogados.com.br/artigos/arranjo-de-pagamentos-o-que-sua-startup-precisa-saber-sobre-esse-sistema Como criar uma Fintech de Pagamentos para Apostas Esportivas? - https://ndmadvogados.com.br/artigos/como-criar-uma-fintech-de-pagamentos-para-apostas-esportivas 23 Sistema de Pagamentos Brasileiro – SPB - https://www.youtube.com/watch?v=1BT2l_REaBI Instituição de Pagamento: como obter autorização do Banco Central | NDM na Prática - https://www.youtube.com/watch?v=F8Vt675A78w Implementing a Regular Expression Engine - https://deniskyashif.com/2019/02/17/implementing-a-regular-expression-engine/ Amazon EC2 P5 Instances | Amazon Web Services - https://www.youtube.com/watch?v=S8JR50sVong Scaling your API with rate limiters - https://stripe.com/blog/rate-limiters Scaling your API with rate limiters - https://gist.github.com/ptarjan/e38f45f2dfe601419ca3af937fff574d Using load shedding to survive a success disaster—CRE life lessons - https://cloud.google.com/blog/products/gcp/using-load-shedding-to-survive-a-success-disaster-cre-life-lessons Top 7 Ways to 10x Your API Performance - https://www.youtube.com/watch?v=zvWKqUiovAM Monitor behind a firewall w/ Private Data source Connect on Grafana Cloud (Grafana Office Hours #04) - https://www.youtube.com/watch?v=9oPe1IICE80 Path to Event-Driven Architecture. Start capturing INTENT! - https://www.youtube.com/watch?v=sPjj8tVKy6M A Simpler Way to See Results - https://www.youtube.com/watch?v=s5S2Ed5T-dc 10 Postgres tips for beginners - https://postgres.ai/blog/20230722-10-postgres-tips-for-beginners 10 Postgres tips for beginners - https://postgres.ai/blog/20230722-10-postgres-tips-for-beginners How Large language Models Work - https://www.youtube.com/watch?v=5sLYAQS9sWQ Kubernetes Node Selector vs Node Affinity vs Pod Affinity vs Tains & Tolerations - https://www.youtube.com/watch?v=rX4v_L0k4Hc why does polymorphism suck? - https://www.youtube.com/watch?v=aq365yzrTVE How to read inference rules - https://cohost.org/prophet/post/2248211-how-to-read-inferenc Learning about distributed systems: where to start? - muratbuffalo.blogspot.com/2020/06/learning-about-distributed-systems.html The Enemy Within: Running Untrusted Code with gVisor - Ian Lewis, Google - https://www.youtube.com/watch?v=1Ib-rfSzDuM I often hear people ask why Kubernetes and Firecracker (FC) can’t just be used together. It seems like an intuitive combination, Kubernetes is popular for orchestration, and Firecracker provides strong isolation boundaries. So why aren’t they compatible yet? Read on 🧵 - https://threadreaderapp.com/thread/1238496944684597248.html Kata containers limitations - https://github.com/kata-containers/kata-containers/blob/main/docs/Limitations.md Kata Containers, gVisor offer more secure container strategies - https://www.techtarget.com/searchitoperations/tip/Kata-Containers-gVisor-offer-more-secure-container-strategies How to deploy Kubernetes with Firecracker? - https://www.padok.fr/en/blog/deploy-kubernetes-firecracker Secure Compute Part 2: gVisor Runtime on EKS - https://www.verygoodsecurity.com/blog/posts/secure-compute-part-2 Firecracker Executor for Apache Mesos - https://www.aventer.biz/en/products/mesos-firecracker/ DynamoDB Users Need To Know About This Update - https://www.youtube.com/watch?v=vizbUmohKKg x64 Virtual Address Translation - https://www.youtube.com/watch?v=W3o5jYHMh8s Podman vs. Docker - https://www.youtube.com/watch?v=Xx588nbshlM So, you want to deploy on the edge? - https://zknill.io/posts/edge-database/ Turbo docs: Concepts - https://docs.turso.tech/concepts What is a warrant canary? - https://www.cloudflare.com/learning/privacy/what-is-warrant-canary/ What is select skip locked for in postgresql 9.5 - https://www.2ndquadrant.com/en/blog/what-is-select-skip-locked-for-in-postgresql-9-5/ SANDBOXING CONTAINERS RUN UNTRUSTED CODE IN A CONTAINER - https://www.scip.ch/en/?labs.20211118 High-throughput stream processing in Rust - https://noz.ai/hash-pipeline/ BLAKE3 Is an Extremely Fast, Parallel Cryptographic Hash - https://www.infoq.com/news/2020/01/blake3-fast-crypto-hash/

July 31, 2023 · 7 min · poorlydefinedbehaviour

Sending logs and traces to Grafana cloud from your Rust application

Sending logs and traces from a Rust app to Grafana cloud Run the Grafana agent Put the Grafana agent config in grafana-agent/agent/agent.yaml. server: log_level: debug logs: configs: - name: default positions: filename: /tmp/positions.yaml scrape_configs: - job_name: varlogs static_configs: - targets: [localhost] labels: job: varlogs app: rust-grafana-metrics-logs-traces __path__: /var/log/app/*log clients: - url: <grafana-cloud-loki-url> traces: configs: - name: default receivers: otlp: protocols: grpc: remote_write: - endpoint: tempo-us-central1.grafana.net:443 insecure: false # only add this if TLS is not required basic_auth: username: <grafana-cloud-tempo-username> password: <grafana-cloud-tempo-password> batch: timeout: 5s send_batch_size: 100 Run the Grafana agent docker run \ -v $PWD/grafana-agent/data:/etc/agent/data \ -v $PWD/grafana-agent/agent.yaml:/etc/agent/agent.yaml \ -v $PWD/log:/var/log/app \ -p 4317:4317 \ grafana/agent:v0.35.0 Dependencies being used Replace axum with whatever dependency you are using. The ones you’ll use no matter the framework are: ...

July 22, 2023 · 3 min · poorlydefinedbehaviour

Thinking about failure, fair-loss links and two generals

It feels like most people are not used to thinking about how things can fail, programming as if things always work as expected is the default modus operandi of most engineers i have talked to. Some examples that come to mind: http requests without handling responses that don’t have status 200, no timeouts, no retries, publishing a message to kafka and them updating a database, having a web client orchestrate a transaction across several systems without thinking: what if the user closes the browser tab? ...

March 28, 2023 · 3 min · poorlydefinedbehaviour

The simple way to avoid deadlocks

A deadlock happens when there’s at least one resource that can be only acquired by one process at a time and there’s a process P1 that is waiting to acquire a resource currently held by a process P2 and P2 is waiting to acquire a resource currently held by P1. The word process does not mean we are talking about just OS processes. There are two processes: Process A wants to acquire a lock on resource 1 and then a lock on resource 2. Process B wants to acquire a lock on resource 2 and then a lock on resource 1. ...

March 25, 2023 · 2 min · poorlydefinedbehaviour

Database anomalies and isolation levels

Anomalies An anomaly or read phenomena can happen when a transaction reads data that may have been modified by another concurrent transaction. Dirty read A dirty read happens when a transaction T1 reads data that has been modified by a concurrent transaction T2 that has not has been committed or rolled back yet. T1 ends up working with stale data if T2 does not commit. T2 starts executing and sets x to a new value, T1 starts executing and reads x, the value of x is the value just set by T2, T2 rolls back, the value of x is not persisted to the database but T1 will move forward with the stale value of x that was written before T2 rolled back. ...

March 24, 2023 · 4 min · poorlydefinedbehaviour