Learn videos
Assessed tutorials and demos, newest first.
-
tutorial · From Videos
How to Post Train Custom Models With Devin and Modal
# Summary: How to Post Train Custom Models With Devin and Modal ## Overview Derek, a deployed engineer at Cognition, demonstrates how to use Devin (an AI software engineer) to post-train custom language models. The video shows Devin autonomously managing a complete machine learning workflow, from training to deployment. ## Key Devin Capabilities Demonstrated **Infrastructure Integration** - Devin can run on third-party GPU infrastructure through "outposts," allowing ML workloads to access GPU acceleration beyond Devin's standard sandboxes - Devin integrates with Modal, an AI infrastructure platform, to access GPU sandboxes for compute-intensive tasks - Devin supports Slack integrations to monitor long-running ML workflows in real-time **Autonomous ML Pipeline Management** - Devin autonomously trained a custom image model (Flux) over a 4-hour period (6 p.m. to 10 p.m.) - The model was trained to generate images of "Devon the otter" (Cognition's mascot) instead of humans when prompted - Devin can be given rules to send status updates at regular intervals (every 15 minutes in this example) - Devin outputs validation images at major training checkpoints to monitor model progress **Model Optimization and Deployment** - Devin automatically selected the optimal checkpoint (step 700 out of 1,000 steps) based on detecting signs of overfitting in later steps - Devin deployed the trained model as a serverless inference function on Modal - Devin integrated the new model into a chat application product, created a pull request, and generated a Vercel preview link - Devin recorded videos and took screenshots to document the testing process **Parallelization and Async Workflows** - Devin supports "sub-sessions" or "sub-agents" that allow parallel work across multiple sessions - Users can step away while Devin works autonomously and receive Slack notifications when assistance is needed ## Context on Post-Training Strategy Cognition believes the future of AI involves post-training open-weight frontier language models with domain-specific evaluations. The company has used this approach internally with its founders and top programmers to create models with frontier-level performance at lower cost than some competitors. ## Technical Details - The example used LoRA (Low-Rank Adaptation), a fine-tuning technique for the image model - Training connected to Weights and Biases for observability metrics - The final model successfully generated otter images instead of human images when tested ## Limitations and Context - The example shown was described as "relatively simple" and "lightweight" with a single training run - Production AI systems typically require experimentation over multiple iterations - Devin supports recursive, self-improving automated post-training pipelines
-
demo · From Videos
Devin, your teammate in Slack
• 🤖 Devin in Slack enables automatic task detection in engineering channels without requiring manual tags or mentions. • 💬 Devin maintains Slack etiquette by replying in threads, avoiding notification spam, and incorporating feedback from other team members. • 🔧 Devin can create tickets, generate pull requests, and produce video demonstrations of fixes within Slack. • ☁️ Devin operates on dedicated cloud infrastructure capable of running applications, testing in browsers, and spinning up end-to-end testing environments. • 📍 Code channels are new dedicated spaces that automatically activate when tasks grow large enough, moving conversations out of engineering threads. • 👀 Code channels provide persistent visibility into Devin's status, pull requests, actions, and cloud session context. • 👥 Multiple team members can join code channels and initiate parallel Devin sessions (e.g., designers creating UI prototypes simultaneously). • 📊 Session artifacts are accessible directly in Slack, including code changes, plans, screenshots, and video demos for confidence in completed work. • 🛠️ Devin in Slack has full access to the same skills and tools available on Devin Cloud, including external integrations like DoorDash. • ✅ Code channels feature is live today for Devin and Slack users.
-
tutorial · From Videos
How to Automate Repetitive Engineering Tasks in Plain English with Devin | 4 Powerful Examples
• 🤖 Devin automates repetitive engineering tasks using plain English descriptions without requiring custom scripts or configuration code. • 🧪 iOS app crash testing automation: Daily 5 a.m. runs that boot the simulator, test all major screens, log and annotate crashes, identify root causes, fix issues, and open pull requests with video recordings attached. • 📊 Database query optimization automation: Daily 5 p.m. runs that connect to Datadog, identify the 10 slowest queries, locate related code, diagnose performance issues, create Linear tickets, and generate pull requests. – Requires OAuth connections to both Linear and Datadog via MCP settings. • 📚 Documentation sync automation: Weekly Sunday runs that review all merged PRs from the previous 7 days and update documentation accordingly. – User must specify the target repository for the automation. • 📦 Dependency update automation: Weekly Sunday runs that identify outdated dependencies, group updates by version type, apply updates, and submit pull requests. • 🔗 Integration capabilities: Automations support connections to Slack, Jira, Linear, Teams, and custom MCPs for connecting to various data sources. • 📋 Template library: Pre-built automation templates available for quick setup with a single click, reducing configuration time.
-
demo · Video
Solving Open Math Problems with 100+ Parallel Devins
Here's a summary of the concrete claims, demos, limitations, and source-worthy details about Devin and Cognition from the video: Devin Product Claims & Demos: Automated Problem Solving:* Devin can be prompted to find and solve open math problems. Parallel Processing:* Devin can spin up and manage over 100 "child sessions" (agents) that work in parallel on different approaches to a problem. "Ultra Mode":* This mode utilizes the best available models (Fable, Soul) for problem-solving. Prompt Variation:* Devin can run the same problem multiple times with slightly different prompts (e.g., "solve X way," "investigate Y") to harness entropy and increase the chance of success. Self-Correction/Iteration:* Devin can re-evaluate, redo tasks, and provide encouraging words to its sub-agents. Environment Improvement:* Devin can optimize its operating environment to increase speed. Content Generation:* Devin can generate follow-up tweets, explanations of solved problems, and graphics for solutions. Formalization & Verification:* Devin can assist in formalizing solutions (e.g., using Lean) and running verification tests. Multi-Agent Communication:* Devin's architecture facilitates communication between agents, with examples of the master agent encouraging child sessions. Sandboxed Environment:* Devin's architecture is described as sandboxed, making it easier to manage multiple agents as "teammates." Public GitHub Repo:* The speaker has made a public GitHub repository containing the formalizations and the process. Cognition Product Claims & Demos: Cloud Agent Product:* Devin is presented as a cloud agent product, which is crucial for enabling long-running, parallel tasks without requiring the user's local machine. "Nerd Sniping" Capability:* The platform can be used to tackle complex, engaging problems. Team of Math Professors:* Cognition has math professors on staff who provided advice on the math problems. Limitations & Nuances: Prompt Engineering is Key:* While Devin automates much of the process, the initial prompt is crucial for directing Devin to find relevant problems. Model Dependence:* The success of problem-solving relies on the underlying AI models (Fable, Soul, GPT-4). "Earth-Shattering" Scale:* The initial problems solved, while novel and previously unsolved, were ranked lower on an "earth-shattering" scale (Richter scale analogy). Potential for Pre-existing Solutions:* One of the solved problems was later found to have a solution in a Git repo from a month prior. Token Limits:* The speaker encountered and had to manage token limits. Verification Importance:* Human verification and collaboration with math experts were necessary to ensure the correctness and avoid embarrassment. AI-Generated Tweets:* Using AI for writing tweets directly can result in boring content; the speaker used it for explanations instead. Not a Replacement for Expertise:* While Devin automates the process, human mathematicians are still needed for deeper understanding and solving truly groundbreaking problems. Exact Source-Worthy Details: Inspiration:* A tweet by "Dimitri" about ChatGPT solving a 30-year-old problem with "gigachad prompting." Initial Prompt:* "Find open math problems we can solve like this together." Methodology:* * Run in "ultra mode" using best models. * Run many agents in parallel. * Run each problem five times with slightly different prompts to harness entropy. * Select the best 10-15 problems. * Focus on counter-examples as a good first step. * "Don't stop. Try a different attack." (Example of agent communication). Verification:* * "Lean formalization" was used to verify solutions. * Adversarial verifications were run. * Solutions were ranked on an "earth-shattering" scale. Problems Solved (mentioned):* * One 30-year-old problem (inspiration). * Several other conjectures: one proved false, one proved true, one refuted. * Five new theorems settled, including a 35-year-old conjecture. Cognition Collaboration:* The speaker mentioned working with math "geniuses" on their team at Cognition. Public Repository:* The speaker is making the repo public with all formalizations. Call for Collaboration:* Seeking a fellow to help solve more unsolved problems.
-
tutorial · From Videos
Cloud Stacked Diffs: Turn One Prompt Into a Stack of Reviewable PRs (with Linear + Devin)
Cloud Stacked Diffs: Turn One Prompt Into a Stack of Reviewable PRs (with Linear + Devin)
-
tutorial · Video
How to Install and Set Up Devin Desktop on Windows 11
How to Install and Set Up Devin Desktop on Windows 11
-
tutorial · Video
How to Build AI Agents That Check Their Own Work | Jared Zoneraich
🔥 Build agents by "letting the model cook": give simple high-level instructions and useful tools, not brittle, ultra-specific prompt hacks. – Tool engineering (choosing which tools the model can use) is the primary lever for capability and longevity. 💡 Align your roadmap with model progress: expect hacks like chain-of-thought prompting to become less important over time. – Do not treat prompt-engineering tricks as long-term defensibility. ⚙️ Rollout strategy: prioritize done over perfect; aim for an 80% working product quickly and invest long tail effort on the final 20%. – Users are often forgiving of probabilistic AI behavior during novel workflows. 🔎 Eval approach: avoid getting stuck on exhaustive deterministic tests before shipping; use smoke tests and iterative production feedback instead. – Agents can be given programmatic sanity checks to validate and iterate on their own outputs. 🤖 Agents should verify and iterate on their own work where possible, using tests, screenshots, or programmatic endpoints as evidence. – Self-evaluation reduces human bottlenecks and accelerates autonomous improvement. ☁️ Cloud agent paradigm: run agents as asynchronous cloud VMs that persist and work independently of a human session. – This enables background work, mobile-triggered jobs, and continuous automation. 🔗 Master–child orchestration: a master agent can spin up multiple child agents (each with its own VM) to parallelize tasks and keep contexts small. – Parallelization speeds large projects and reduces per-agent context-window pressure. 🧩 Use cases for fan-out agents: large code migrations, parallelized testing, focused single-responsibility tasks, and breaking monolithic jobs into verifiable chunks. – Each child agent can write, run, test, take screenshots, and produce PRs independently. 👥 Multiplayer workflow: support human + agent collaboration and team-shared skills, macros, playbooks, and indexed docs to capture knowledge and speed onboarding. – Indexing repos and auto-generating docs provides immediate ROI on large or poorly-documented codebases. 🏢 Product fit: agent harnesses that prioritize verifiable code, testing, and brownfield engineering are attractive to enterprises where correctness matters. – These tools are especially useful for migrations, test coverage, and legacy modernization. 💸 Cost model: evaluate spending by ROI, not token count; route simple tasks to cheaper models and complex tasks to stronger models to balance cost and value. – A harness should optimize model selection for task-fit rather than incentivize token maximization. 🌐 Market view: building a great coding agent is a core, high-leverage problem with many downstream applications, and the market may bifurcate into hardcore engineering tools and short-form "play" tools. – Non-technical agent use cases will continue to grow alongside coding-focused products. 🧭 Long-term thesis: many artifacts (raw code, slides, simple UIs) may become second-order outputs produced by higher-level agent interactions or APIs. – Design products for agent-first access patterns and discoverability rather than only human-first UI.