🤖 OmbuLabs.ai

Looking forward to seeing old and new #RubyFriends at @sfruby.com next month! Who's going?! #SFRuby #Ruby

default_scope doesn't just filter queries. It quietly sets attributes on new records, and unscoped strips every scope, not just the one you wanted gone. go.upgradejs.com/5so #RubyOnRails #TechnicalDebt

One gem was adding ~850ms to every rails console and rake task. Bumbler shows which requires are slow so you can stop loading them by default. go.ombulabs.com/c07 #RailsPerformance

Coverage-based step scoring diagnosed agent failures with 0.91 accuracy at hop 1. At hops 2 and 3 it dropped to 0.00. A counterfactual probe got 0.67 where coverage got zero: go.upgradejs.com/ycl #AIEngineering

New models ship roughly every 11 days, so don't build your product around today's limitations. Being early is a bet, not a free advantage. go.upgradejs.com/ggv #EnterpriseAI

Jev's headline context is 64k tokens, but your state plus your longest single question has to fit in 32k. Read the fine print before you design around it: go.upgradejs.com/frq #AIEngineering

A monthly client report that took most of a morning now takes less than half that: one Claude Code skill, one read-only MCP server over Phoenix, Sentry, and the product DB. go.upgradejs.com/gtd #MCP #ClaudeCode

Not every problem needs AI. We start by finding the right problem, then pick the tool: a model, a RAG pipeline, or plain old tech debt cleanup. go.upgradejs.com/p86 #AIConsulting

Fix cohesion first and coupling tends to fix itself. Most of SOLID is those two ideas in different clothes. go.ombulabs.com/f5n #SoftwareDesign

Claude now edits Google Docs, Sheets, and Slides from a sidebar, and your existing connectors and skills come along. Start in "Ask before edits" mode. go.upgradejs.com/nkd #ClaudeAI

Not every problem needs AI. We start by finding the right problem, then pick the tool: a model, a RAG pipeline, or plain old tech debt cleanup. go.upgradejs.com/p86 #AIConsulting

Name your span attributes with the OpenTelemetry gen_ai conventions, like gen_ai.request.model. Same names across every service is what lets you correlate one trace end to end instead of guessing. go.upgradejs.com/rey #OpenTelemetry

Judging an AI design system by the final image tells you almost nothing. Evaluate the workflow: did retrieval pull the right reference images, did the business rules actually apply. go.upgradejs.com/zex #AIEngineering

Is your team using Claude at work? Is your org's data safe? Join our next webinar on Claude and privacy. Free, 45 min, Thu Oct 22 at 11am ET. RSVP: www.ombulabs.ai/webinar/safe...

Tracing an LLM app is a cost tool too. Token usage per model, per user, per feature means you optimize spend instead of finding out at the invoice. One span in our example burned 8,814 tokens. go.upgradejs.com/umw #LLMOps

Incompatible Rubies, three version managers, bundle install dying on a C extension. Containers skip OS emulation and lean on chroot, namespaces, and cgroups, which is why one boots while a VM is still allocating RAM. go.ombulabs.com/9oi #LegacyCode

Jev returns only a typed answer: a Choice, a Score, or a probability. No free text means it cannot hallucinate an option you never defined. go.upgradejs.com/46s #AIEngineering

Judge panel unanimity in agent evals hit 18.13x false consensus lift on one evidence type. Agreement was correlated error, not independent validation. go.upgradejs.com/r18 #AIEngineering

Ask a narrow question, not a mission. Jev says its 0.8-probability answers land right about 80% of the time, and it will not invent an option outside your schema. go.upgradejs.com/g1u #AIEngineering

C2PA metadata says which software made a file, not who made it. And screenshot a marked image and the manifest is gone entirely. Worth knowing before you build policy on it. go.upgradejs.com/shc #AIPolicy

Step-level agent grading caught 91% of injected faults at hop one, then zero at hop two. Later steps quietly patch earlier mistakes. go.upgradejs.com/sv9 #AIEngineering

An agent is a loop, not a chatbot. Reach for one only when a task is frequent, too varied for fixed rules, and safe enough to get wrong sometimes. go.upgradejs.com/395 #AIAgents

Skip the 1-10 rubric. Binary pass or fail gives an LLM judge a cleaner signal, and it scales the same human calibration that built the eval set. go.upgradejs.com/oci #LLMOps

Sandi Metz's limits are blunt and useful: classes under 100 lines, methods under 5, no more than 4 parameters. Ask a class what its job is, and if the answer needs an "and", you found the split. go.upgradejs.com/7oh #Refactoring

String interpolation inside a where clause is the classic Rails SQL injection. Pass the value as a bind parameter instead, then let Brakeman check the rest of the app. go.upgradejs.com/7ft #RubyOnRails

A Rails 4.2 app nobody wants to touch is a good candidate for a container. Start from Ubuntu, set DEBIAN_FRONTEND=noninteractive so Postgres stops prompting mid-build, and the whole setup ritual turns into a file. go.upgradejs.com/8lc #Docker #Rails

Give retrieval its own span: the query you sent, the documents that came back. Then a bad answer stops being a mystery and becomes one question, retrieval or generation. go.upgradejs.com/y86 #RAG

We rebuilt our case studies page. Same four projects, far easier to read now. www.ombulabs.ai/case-studies...

Sort your AI tools into three buckets: annoying if it dies, slow if it dies, revenue stops if it dies. Only the third needs a backup provider now. go.upgradejs.com/iga

An LLM judge earns its keep once it agrees with your human labels 80 to 90 percent of the time. Below that you are automating noise. go.upgradejs.com/8nk

Pick one sanctioned low risk use case and let the whole team practice there first. Permission plus a boundary beats a policy nobody reads. go.upgradejs.com/n77

AI discovery starts with interviews, not tools. You cannot automate a workflow nobody described. www.ombulabs.ai/ai-strategy-...

Traced your LLM app? The same spans that debug a bad run are the raw material for an evaluation system. We just pointed our tracing post at the follow-up. go.upgradejs.com/3zn #LLMOps

A model reviews our time entries and surfaces only the ones worth a second look. That split is the design work. www.ombulabs.ai/case-studies...

Not everything deserves a span. LLM calls, tool calls, retrieval, prompt version: yes. Generic infra spans: no, they bury the four things you actually need. go.upgradejs.com/don #LLMOps

Teams running agents in production do not pick one top-ranked model. They route different tasks to different models, and they stop upgrading once task completion stops moving. go.upgradejs.com/v21 #AIAgents

Classify and drop non-events early and the expensive part of an AI pipeline never sees them. Some errors still slip through, which is why a human reviews the flagged ones. go.upgradejs.com/3ac #DataEngineering

You do not have to hand-roll spans. One package plus an OTLP endpoint gets a full LLM trace into Phoenix or Langfuse without touching your agent code. go.upgradejs.com/0aq #LLMOps

30 to 40% of LLM-generated selectors fail on the first try. Blind retries do not help. Classify the failure (too narrow, too broad, wrong region) and cap it at 3 attempts before a human looks. go.upgradejs.com/z0w #WebScraping

An assessment is only useful if someone builds what it recommends. Ours ends with a roadmap, a security framework, and config files your team owns, ready to hand straight to implementation. go.upgradejs.com/9qc #AI

Sometimes the honest answer is that a script, a form change, or a better report beats a model. Deciding not to build the AI feature is a real outcome, not a failure. go.upgradejs.com/dlv #AIStrategy

The control surface for a ReAct agent is the system prompt and the tool schema. Ours refuses to save until the post has both a title and an image, and the tool rejects anything under 20 characters. go.upgradejs.com/xhd #AIAgents

Not every entry needs a model. Hard rules handle the common cases, a binary classifier takes the ambiguous ones, and misfiled billable time dropped 95%. go.upgradejs.com/69p #MachineLearning

Starting today, Sept 9 11am ET: free Claude webinar. www.ombulabs.ai/webinar/safe...

Tomorrow, Sept 9 11am ET: free Claude webinar. www.ombulabs.ai/webinar/safe...

A regression model is the easy half. The other half is FastAPI, MLflow, and an API that lets a C# platform call it. That is what turns research into something a team can actually run. go.upgradejs.com/fo8 #MachineLearning

Happening tomorrow: Free webinar on safe #Claude rollouts for small businesses. www.ombulabs.ai/webinar/safe... #AI #Training #Webinar

Free webinar Wed 9/9, 11am ET: safe Claude rollout www.ombulabs.ai/webinar/safe...

Nobody ships a benchmark score. The metrics that predict a useful agent are human acceptance rate, cost per task, and failure recovery rate. go.upgradejs.com/77o #AIAgents

Every LLM call in this pipeline returns structured output, not free text, and the agent loop is capped at a 7 minute budget. Hallucinations never reach the database. go.upgradejs.com/20a #DataEngineering