Miguel Piedrafita ✨

Miguel Piedrafita ✨

@miguel.build

23. serial builder, purple-haired developer @world.org, core @ConstitutionDAO. friendly ambitious nerd.

754 followers582 following891 posts

Miguel Piedrafita ✨
@miguel.build

thought i'd built a very simple iOS app during Christmas, then spent most of the time on this screen will probably not finish the app (HomeKit APIs are worse than I expected), but came out pretty cool!

Miguel Piedrafita ✨
@miguel.build

I may need to bring this back at some point because holy shit would that be useful when planning this video

Miguel Piedrafita ✨@miguel.build

✨ I've been writing some scripts for videos I wanna record recently, but was unhappy with the available software. So I made a concept for an app that allows you to define both the script and the scenes in the same place. ⚡️ Is this something you'd use?

Miguel Piedrafita ✨
@miguel.build

if anyone's curious: when running chrome in headless mode it only allocates enough GPU space for drawing around 6000px x 6000px for any website meaning if you have a really long website (like i do), there's no way to get a full screenshot natively

Miguel Piedrafita ✨
@miguel.build

thought I'd try to figure out exactly how rewind.ai works today. the tl;dr is: - screenshot your screen every ~2s, store all the screenshots as a video file (HEVC, so we get compression for free) - constantly monitor the app the user's in, and save it to an encrypted sqlite db

View full thread
Miguel Piedrafita ✨
@miguel.build

the encrypted database lives at ~/Library/Application Support/com.memoryvault.MemoryVault/db-enc.sqlite3 btw password for mine was soiZ58XZJhdka55hLUp18yOtTUTDXz7Diu7Z4JzuwhRwGG13N6Z9RTVU1fGiKkuF but not sure if it's hardcoded or generated on first launch

Miguel Piedrafita ✨
@miguel.build

here's a weird thing: I can find references in the code to ask rewind also supporting claude and llama via a different backend hosted on cloudflare workers, but as far as I can tell there's no way to actually use it in the app

Miguel Piedrafita ✨
@miguel.build

thought I'd try to figure out exactly how rewind.ai works today. the tl;dr is: - screenshot your screen every ~2s, store all the screenshots as a video file (HEVC, so we get compression for free) - constantly monitor the app the user's in, and save it to an encrypted sqlite db

View full thread
Miguel Piedrafita ✨
@miguel.build

for meeting transcription/recording, it detects when you're joining one *somehow* (looking into that now) then records system audio for the duration of the meeting and transcribes it locally using whisper-base then sends it over for the summarization to... chatgpt? i think?

Miguel Piedrafita ✨
@miguel.build

yep, it's chatgpt, specifically gpt-4o-mini-2024-07-18 Ask Rewind is also powered by chatgpt. on an example query, it first calls a `query_history` function to get more information about the captured activity, then streams back the actual response

Miguel Piedrafita ✨
@miguel.build

thought I'd try to figure out exactly how rewind.ai works today. the tl;dr is: - screenshot your screen every ~2s, store all the screenshots as a video file (HEVC, so we get compression for free) - constantly monitor the app the user's in, and save it to an encrypted sqlite db

Miguel Piedrafita ✨
@miguel.build

- for each screenshot, also attempt to OCR the text on the screen for indexing, as far as I can tell it's just using VNRecognizeTextRequest - also, for certain apps (browsers), try to get the current url (via `getWindowProperties` as far as I could see)

Miguel Piedrafita ✨
@miguel.build

cool things I built last month for @world.org: 🪪 Credentials – using your phone's NFC scanner to read passport data and validate its authenticity, without sending any personal info to a server 👤 DeepFace – letting you authenticate streams of video (or photos) as real

Miguel Piedrafita ✨
@miguel.build

the  path I took at 16 to get to where I am now doesn't exist anymore i really dislike the state of twitter nowadays and have been posting a lot less because of this only recently I realized that makes me part of the problem in a way anyways, lots more tweets soon

Miguel Piedrafita ✨
@miguel.build

if anyone else is struggling with this (enjoying twitter like you did before) I've found that, while aggressive, muting anyone you come across expressing unnecessarily negativity (judge by yourself) helps a lot!

Miguel Piedrafita ✨
@miguel.build

want to build iOS apps with @OpenAI's new Realtime API? Here's a Swift package that handles all the hard stuff (like websockets) so you can focus on your app 😁 (will also add support for managing conversations and voice for you in the coming days) https://t.co/Gp24w9NW3k

github.comGitHub - m1guelpf/swift-realtime-openai: A modern Swift SDK for OpenAI's Realtime APIA modern Swift SDK for OpenAI's Realtime API. Contribute to m1guelpf/swift-realtime-openai development by creating an account on GitHub.
Miguel Piedrafita ✨
@miguel.build

if you wanna deploy @OpenAI's new realtime APIs to mobile/web apps, you'll need a proxy websocket server. this lets you handle auth, rate limits, private info, and not leak API keys!! here's a simple 🦀 lib to build & deploy one of these in 5 minutes. https://t.co/nIyZB8FzyS

github.comGitHub - m1guelpf/openai-realtime-proxy: Safely deploy OpenAI's Realtime APIs in less than 5 minutes!Safely deploy OpenAI's Realtime APIs in less than 5 minutes! - m1guelpf/openai-realtime-proxy
Miguel Piedrafita ✨
@miguel.build

i keep seeing people in the tl optimizing for the wrong things avoid the signaling, don't try to be the main character, engagement won't make you happy build something that makes you proud of yourself, make some friends, be kind to everyone

Miguel Piedrafita ✨
@miguel.build

recently got a Blackmagic camera, which doesn't have continuous auto-focus. you can fix this by buying a $1k LiDAR kit from DJI, which also requires you to have a motor physically turn the focus ring on the lens. time to bust out an ESP32 i guess 🫠

Miguel Piedrafita ✨
@miguel.build

It's incredibly safe since all GitHub Actions does is tell your server to deploy, but you still get real-time progress attached to your commits. It's also super easy to get started, just run the binary or the Docker image in your server and add the pre-made action to your repo.

Miguel Piedrafita ✨
@miguel.build

All of this brought to you by not being able to sleep on Saturday and getting nerd-sniped by infrastructure work instead. But hey, four crates after (🦀 is fun) I ended up with a pretty handy open-source envoyer thingy 😁 now, back to actually building the app I wanted to build

Miguel Piedrafita ✨
@miguel.build

been struggling to write code over the past few weeks today, i turned on code coverage view on vscode (the thing where it shows you which lines get hit by tests), and slowly making all the red lines turn green ended up being a great motivator

Miguel Piedrafita ✨
@miguel.build

been playing around with server-side swift and @codevapor recently, and slowly building some nice flows around it ✨ for example, here's a lil package that lets you name your routes (+ resolve them from javascript, thanks to @tightenco's ziggy library) https://github.com/m1guelpf/ziggy-vapor

github.comGitHub - m1guelpf/ziggy-vapor: Named routes for your Vapor app, both on Swift and on Javascript!Named routes for your Vapor app, both on Swift and on Javascript! - m1guelpf/ziggy-vapor
Miguel Piedrafita ✨
@miguel.build

i strongly believe most of the workflows and tools we use daily have 10-100x improvements hidden along the gradient descent very inspiring to see people continue to venture up the curve, regardless of how crazy it might seem from the outside