Jekyll2026-02-05T10:00:46+00:00https://blog.flurdy.com/feed.xmlflurdy’s rantingThe ramblings and ranting of Ivar Abrahamsen at flurdy.com. Contain ideas, ranting at innocents, blinkered sporting opinions, tech bable, and probably not enough to be interesting.Mob Together: When AI Joins the Team2026-02-04T12:52:00+00:002026-02-04T12:52:00+00:00https://blog.flurdy.com/2026/02/mob-together-when-ai-joins-the-team<h2 id="what-is-mob-programming">What is mob programming?</h2> <p>Mob programming is simple: the whole team works on the same thing, at the same time, in the same space, on the same computer.</p> <p>One person types (the “driver”), while everyone else navigates - discussing, suggesting, reviewing in real-time. The driver role rotates regularly.</p> <p>It sounds inefficient. Five people doing one person’s work? But something counterintuitive happens: you go faster, not slower.</p> <ul> <li>Decisions happen immediately instead of waiting for reviews</li> <li>Knowledge spreads across the team naturally</li> <li>Bugs get caught as they’re written</li> <li>No merge conflicts, no integration problems</li> </ul> <h3 id="not-meetings-not-chaos">Not meetings, not chaos</h3> <p>Mob programming is not a meeting. You’re producing code the whole time. It’s also not chaos - there’s structure, rotation, and focus.</p> <p>Think of it as continuous code review with immediate implementation.</p> <h3 id="life-happens-code-flows">Life happens, code flows</h3> <p>One underrated benefit: flexibility. People take breaks, step out for meetings, leave early, go on holiday - and the code keeps flowing. No one is a bottleneck because everyone shares the context.</p> <p>Compare this to solo work where one person’s absence means their work stops, their PRs wait for review, and their knowledge stays locked in their head.</p> <h3 id="fewer-rabbit-holes">Fewer rabbit holes</h3> <p>Like pair programming, mob programming naturally prevents waste. When you’re about to spend two hours on a “nice to have” abstraction, someone asks: “Do we actually need this?”</p> <p>Rabbit holes get spotted early. Unnecessary code never gets written. The group maintains focus on what matters.</p> <p class="post-photo-narrow"><a href="/img/posts/2026/02/silos-barn.jpg"><img src="/img/posts/2026/02/silos-barn-small.jpg" alt="Silos" /></a></p> <p class="post-photo-license"><em><a href="https://www.freepik.com/free-photo/wide-shot-concrete-silos-barns-grass-field-surrounded-with-trees_7908980.htm">Image by Freepik</a></em></p> <h2 id="the-solo-ai-trap">The solo AI trap</h2> <p>AI coding assistants are transforming how we work. But here’s what often happens:</p> <p>Each developer on your team works alone with their AI. They each:</p> <ul> <li>Interpret requirements differently</li> <li>Make different architectural choices</li> <li>Build overlapping or conflicting solutions</li> <li>Create code the others don’t understand</li> </ul> <p>You end up with multiple teams inside one team. Five developers, five directions, five interpretations. The AI amplifies individual productivity - and individual divergence.</p> <blockquote> <p>Solo developers with AI agents are highly productive. But productive at what?</p> </blockquote> <p>Code gets written fast. Integration takes forever. Debugging someone else’s AI-generated code is painful. The team fragments.</p> <h2 id="the-mob--ai-sweet-spot">The mob + AI sweet spot</h2> <p>Here’s the insight: AI agents don’t replace mob programming - they supercharge it.</p> <p>A small mob (4-5 people) working together with AI agents:</p> <ul> <li><strong>Shared context</strong> - Everyone sees the same prompts, specs, and AI responses</li> <li><strong>Collective prompt refinement</strong> - Four minds crafting better prompts than one</li> <li><strong>Parallel execution</strong> - Run multiple agents on different tasks, all aligned</li> <li><strong>Real-time review</strong> - AI output gets immediate human scrutiny from multiple perspectives</li> </ul> <p>The mob provides what AI lacks: judgment, context, and coherence.</p> <p class="post-photo-narrow"><a href="/img/posts/2026/02/small-mob-robots.png"><img src="/img/posts/2026/02/small-mob-robots-small.png" alt="Small Robot mob" /></a></p> <p class="post-photo-license"><em>AI generated image with Nano Banana. CC-by</em></p> <h2 id="why-smaller-mobs-work-better-with-ai">Why smaller mobs work better with AI</h2> <p>Traditional mob programming sometimes used larger teams (8-9 people). With AI, smaller works better:</p> <h3 id="more-active-participation">More active participation</h3> <p>With 4-5 people, everyone stays engaged. No one becomes a passive observer. Each person can meaningfully contribute to prompt crafting and output review.</p> <h3 id="parallel-agent-orchestration">Parallel agent orchestration</h3> <p>A small mob can effectively coordinate multiple AI agents working simultaneously:</p> <ul> <li>One agent implementing a feature</li> <li>Another writing tests</li> <li>A third updating documentation</li> <li>An agent can be tasked with planning etc</li> </ul> <p>More people doesn’t help here - you need coordination, not headcount.</p> <h3 id="faster-rotation">Faster rotation</h3> <p>Smaller groups rotate the driver role more frequently. Everyone stays sharp and engaged.</p> <h2 id="the-ideal-team">The ideal team</h2> <p>I believe small mob teams with shared AI collaboration represent the most productive team structure we’ve found yet.</p> <p>Not individuals racing ahead with their own AI agents. Not large teams with coordination overhead. But 4-5 people, thinking together, prompting together, building together.</p> <h3 id="mixed-backgrounds-shared-goal">Mixed backgrounds, shared goal</h3> <p>The ideal mob isn’t five backend developers. It’s a mix - someone with frontend experience, someone who knows infrastructure, someone strong on architecture, maybe someone closer to product.</p> <p>This diversity is a feature, not a problem. When you hit a CSS issue, the frontend person leads. When deployment questions arise, the infra person steps in. Everyone learns from each other constantly. Knowledge spreads without scheduled “knowledge transfer” sessions.</p> <p>You don’t need experts in everything. You need enough overlap to communicate and enough diversity to cover the problem space.</p> <h3 id="flexible-focus">Flexible focus</h3> <p>Not everyone needs to stare at the same screen every moment. While the core group drives the main story forward, someone can branch off to investigate:</p> <ul> <li>Research an API the team will need</li> <li>Spike a potential approach</li> <li>Check documentation or find examples</li> </ul> <p>They stay connected - same call, same shared context - and rejoin the main flow when ready. The investigation feeds back into the mob’s work rather than becoming a separate track.</p> <p>This flexibility means the mob moves fast on the main path while still handling the side quests that would otherwise block progress.</p> <p class="post-photo-narrow"><a href="/img/posts/2026/02/cogs.jpg"><img src="/img/posts/2026/02/cogs-small.jpg" alt="Cogs" /></a></p> <h3 id="the-sum-is-greater">The sum is greater</h3> <p>The AI handles the typing speed. The mob handles the thinking speed.</p> <p>Four people with different perspectives, reviewing AI output together, catching issues the AI missed, asking “but what about…” - this produces better results than any individual could achieve, no matter how good their AI tools.</p> <h2 id="how-it-works-in-practice">How it works in practice</h2> <h3 id="shared-specifications">Shared specifications</h3> <p>The mob starts by agreeing on specs and acceptance criteria together. This alignment - which solo developers skip - means AI agents receive consistent, well-thought-out prompts.</p> <h3 id="collaborative-prompting">Collaborative prompting</h3> <p>One person drives, but everyone contributes to the prompt. “What if we ask it to…” becomes a team activity. The resulting prompts are clearer and more complete.</p> <h3 id="parallel-execution-with-oversight">Parallel execution with oversight</h3> <p>The team can run multiple agents on related tasks. Because everyone has shared context, anyone can review any agent’s output. No knowledge silos.</p> <h3 id="immediate-course-correction">Immediate course correction</h3> <p>When an AI goes in the wrong direction, the mob catches it immediately. No wasted hours following a bad path alone.</p> <h2 id="the-fragmentation-problem">The fragmentation problem</h2> <p>When developers work solo with AI, you get:</p> <ul> <li><strong>Architectural drift</strong> - Each person’s AI makes different structural choices</li> <li><strong>Style inconsistency</strong> - Code looks like it was written by different teams (because it was)</li> <li><strong>Knowledge silos</strong> - Only one person understands each piece</li> <li><strong>Integration nightmares</strong> - Merging divergent AI-assisted work is painful</li> </ul> <p>A mob stays aligned by definition. There’s one codebase direction, one architectural vision, one understanding.</p> <p class="post-photo-narrow"><img src="/img/posts/2026/02/cabin-work-small.jpg" alt="Cabin work" /></p> <p class="post-photo-license"><em><a href="https://stockcake.com/i/remote-work-paradise_3076106_1423399">Photo by StockCake</a></em></p> <h2 id="remote-mob-programming">Remote mob programming</h2> <p>You don’t need a physical room. Remote mob programming works surprisingly well:</p> <ul> <li>Screen sharing (driver shares their screen)</li> <li>Video call for discussion</li> <li>Shared documentation for specs and context</li> <li>Rotation timer to keep things moving</li> </ul> <p>In some ways, remote is better - everyone has the same view, recordings are easy, and introverts often participate more freely.</p> <h3 id="beyond-one-driver">Beyond one driver</h3> <p>Tools like VS Code Live Share change the dynamic. Multiple people can edit the same file simultaneously - one person leads, but others can jump in to fix a typo, add a comment, or demonstrate an idea.</p> <p>The strict “one driver” rule relaxes into “one person in charge, flexible for input.” This can accelerate things further - a navigator spots a missing semicolon and just fixes it instead of dictating the correction.</p> <p>Traditional mob purists might object, but the goal is flow, not rules.</p> <h2 id="getting-started">Getting started</h2> <p>If you haven’t tried mob programming:</p> <ol> <li><strong>Start small</strong> - One afternoon, one feature, 3-4 people</li> <li><strong>Set a rotation timer</strong> - 10-15 minutes per driver</li> <li><strong>Pick something meaty</strong> - A real feature, not a trivial task</li> <li><strong>Include the AI</strong> - Have your coding assistant open and visible to all</li> </ol> <p>The first session will feel awkward. That’s normal. By the third session, you’ll find your rhythm.</p> <h3 id="what-youll-notice">What you’ll notice</h3> <ul> <li>Fewer “how does this work?” questions later</li> <li>Less time in code review (it already happened)</li> <li>More consistent code style</li> <li>Better prompt engineering as a team skill</li> </ul> <h2 id="try-it">Try it</h2> <p>If your team uses AI assistants, try mobbing for a week. You might find that working together - truly together - is faster than working apart.</p> <blockquote> <p>One team, many agents, shared understanding</p> </blockquote> <hr /> <h2 id="resources">Resources</h2> <h3 id="books">Books</h3> <ul> <li> <p><a href="https://www.amazon.com/Software-Teaming-Programming-Whole-Team-Approach/dp/B0BLG1QTYK">Software Teaming: A Mob Programming, Whole-Team Approach</a> - Woody Zuill and Kevin Meadows. The definitive guide, now in its second edition with material on remote mob programming.</p> </li> <li> <p><a href="https://pragprog.com/titles/mpmob/code-with-the-wisdom-of-the-crowd/">Code with the Wisdom of the Crowd</a> - Mark Pearl. Practical guide to getting started with mob programming.</p> </li> </ul> <h3 id="podcasts">Podcasts</h3> <ul> <li> <p><a href="https://mobmentalityshow.podbean.com/">The Mob Mentality Show</a> - Chris Lucian and Austin Chadwick discuss mob programming with guests. Over 300 episodes covering everything from getting started to advanced patterns.</p> </li> <li> <p><a href="https://www.mountaingoatsoftware.com/agile/podcast/33-mob-programming-with-woody-zuill">Mob Programming with Woody Zuill</a> - Mountain Goat Software podcast episode with the originator.</p> </li> </ul> <h3 id="websites">Websites</h3> <ul> <li><a href="https://mobprogramming.org/">Mob Programming</a> - Woody Zuill’s original site with resources and sample book chapters</li> <li><a href="https://woodyzuill.com/">Woody Zuill</a> - Workshops, talks, and coaching on software teaming</li> </ul>flurdyWhat is mob programming?YAGNI × 100 with AI2026-02-02T16:00:00+00:002026-02-02T16:00:00+00:00https://blog.flurdy.com/2026/02/yagni-100-with-ai<h2 id="you-arent-gonna-need-it">You Aren’t Gonna Need It</h2> <p>YAGNI - don’t build features until you actually need them. History taught us that if we added all the features we thought we would need, projects would be late, over budget, or never finished at all. Most of those extras were rarely needed in the end.</p> <p>Release early with basic features and see if more polish is actually needed. That way you ship on time and budget, but also gain real feedback on what works. You may add a few more features later - but only a tiny subset of what you thought you needed.</p> <h3 id="the-old-friction-was-our-friend">The old friction was our friend</h3> <p>Before AI, there was natural friction. Want to add a caching layer? That’s a few hours. An abstraction for three database backends? A day or two. A plugin system for your internal tool? Block out a week.</p> <p>This friction served as a filter. When building something unnecessary costs significant time, you think twice. You weigh the trade-offs. You ask yourself: “Do I really need this?”</p> <p>YAGNI saved days, weeks, months. It saved projects, businesses and careers.</p> <h2 id="the-new-world-the-ai-world-no-friction">The New World, the AI World, No Friction</h2> <p>With AI agents, that calculation changes dramatically as AI removes the friction. Now you can:</p> <ul> <li>Generate a complete caching layer in minutes</li> <li>Build abstraction layers with a few prompts</li> <li>Create plugin architectures before lunch</li> </ul> <p>The friction is gone. And with it, our natural defense against over-engineering.</p> <blockquote> <p>AI makes it so easy to write code that we forget to ask if we should</p> </blockquote> <p><img src="/img/posts/2026/02/tip-of-the-iceberg-small.png" alt="Tip of the iceberg" /></p> <h2 id="the-hidden-costs-remain">The hidden costs remain</h2> <p>But in fact the real costs was not just the development time, more code means:</p> <ul> <li> <p><strong>Maintenance burden</strong></p> <p>Code that exists must be maintained, updated, and understood</p> </li> <li> <p><strong>Bug surface area</strong></p> <p>More code means more places for bugs to hide</p> </li> </ul> <p>But also on individual level:</p> <ul> <li> <p><strong>Cognitive load</strong></p> <p>Every abstraction adds complexity for the next developer (including future you)</p> </li> <li> <p><strong>Onboarding friction</strong></p> <p>New team members must understand code that serves no purpose</p> </li> </ul> <h3 id="not-just-code">Not just code</h3> <p>Even with super charged AI development, if you keep adding features fast, each one will still delay a project more and more:</p> <ul> <li> <p><strong>Testing overhead</strong></p> <p>That unnecessary feature still needs tests.</p> <p>And often manual QA, or at least some manual UX test to ensure it has been done.</p> </li> <li> <p><strong>Documentation</strong></p> <p>More features, means more documentation and training</p> </li> </ul> <p class="post-photo-narrow"><a href="/img/posts/2026/02/complicated-house-hooba-design.jpg"><img src="/img/posts/2026/02/complicated-house-hooba-design-small.jpg" alt="Complicated house" /></a></p> <p class="post-photo-license"><em><a href="https://www.dezeen.com/2024/12/10/marbella-residential-complex-hooba-design-group-iran/">Image by Hooba Design</a></em></p> <h2 id="yagni--100-to-avoid-ai-bloat">Yagni × 100 to avoid AI bloat</h2> <p>With AI agents ability to create features at such rapid rate, you will need a strict discipline to keep the project on course and apply YAGNI all the time to stop the bloat.</p> <p>AI increases output, not understanding. If you build five extra features you do not need, you also create five extra surfaces for bugs, support, and distraction.</p> <h3 id="the-discipline-shift">The discipline shift</h3> <p>In the old world, we needed discipline to push through and finish features.</p> <p>In the AI world, we need discipline to stop ourselves from building features.</p> <p>The AI suggests adding configuration options, extensibility points, error handling for scenarios that won’t happen, abstractions for problems you don’t really have. And because it’s so easy, the instinct is to say yes.</p> <h2 id="before-you-build">Before you build</h2> <p>Before asking your AI assistant to build something, ask yourself:</p> <ol> <li><strong>Is this solving a real problem I have today?</strong> Not a hypothetical future problem. Today.</li> <li><strong>What’s the simplest solution?</strong> AI loves to suggest comprehensive solutions. Push back.</li> <li><strong>Can I delete this later?</strong> If the code is deeply integrated, think harder about whether you need it.</li> <li><strong>Would I write this by hand?</strong> If the effort would have stopped you before, maybe that was wisdom.</li> </ol> <h3 id="resist">Resist</h3> <blockquote> <p>The best code is still no code. The second best is the minimum code that solves the actual problem.</p> </blockquote> <p>AI makes it tempting to skip straight to comprehensive solutions. Resist.</p>flurdyYou Aren’t Gonna Need ItAI in SDLC = the End of Outsourcing?2025-08-11T22:00:00+00:002025-08-11T22:00:00+00:00https://blog.flurdy.com/2025/08/ai-end-of-outsourcing<p>Is outsourcing development business at risk? By AI agents? Does it replace the reasons people outsource?</p> <h2 id="why-outsource">Why outsource?</h2> <p>Companies outsource projects, teams or whole departments for several reasons.</p> <h3 id="what-are-the-benefits">What are the benefits?</h3> <h4 id="capacity">Capacity</h4> <p>Ability to run more projects in parallel than they have to employ and train staff for.</p> <h4 id="ability">Ability</h4> <p>Some projects may require skills the company does not have or want to have.</p> <h4 id="flexibility">Flexibility</h4> <p>The ability start and cancel projects without retaining permanent employees. Or change requirements and skills required on a whim.</p> <p>No-one sheds tears for a consultant if a project gets canned.</p> <p class="post-photo-narrow"><img src="/img/posts/2025/08/office-space-small.jpg" alt="Outsourced office" /></p> <p class="post-photo-license"><em><a href="https://stockcake.com/i/busy-office-space_481743_105167">Photo by Stockcake. Public Domain</a></em></p> <h4 id="cost">Cost</h4> <p>Local consultants costs more per hour but you do not pay for them all the time. Nor any of the benefits etc. If the project is not started, is paused or when it is finished that costs is 0.</p> <h4 id="risk">Risk</h4> <ul> <li>Reducing the risk of enlarged employee salary budget.</li> <li>Risk of shortage of staff means running out of time to launch project.</li> <li>Risk of the company falling behind by not keeping up with features.</li> </ul> <h3 id="why-outsource-to-cheaper-countries">Why outsource to <em>cheaper</em> countries?</h3> <p>And more importantly many outsourced projects is outsourced to much cheaper countries, where maybe a whole team cost less than one permanent employee.</p> <p>That means either</p> <ul> <li>the scale and capacity of the companies projects can be much larger</li> <li>Or that the budget can be much smaller.</li> </ul> <p>In theory.</p> <p class="post-photo-narrow"><img src="/img/posts/2025/08/overwhelmed-small.jpg" alt="Overwhelmed manager" /></p> <p class="post-photo-license"><em><a href="https://stockcake.com/i/overwhelmed-office-worker_1342777_969535">Photo by Stockcake. Public Domain</a></em></p> <h2 id="fallacies-of-outsourcing">Fallacies of outsourcing</h2> <p>There are many downsides to outsourcing: (Though that is really out of scope for this article.)</p> <h4 id="company-culture-and-harmony">Company culture and harmony:</h4> <p>Often the companies employees feel marginalised by the focus the new teams and the influx of the new co-workers. Or the outsourcing company and consultants have completely different norms and behaviour of the original staff.</p> <h4 id="productivity">Productivity</h4> <p>Bottle-necks and communication overheads may mean the company actually ship less. Timezone differences, culture misunderstandings, etc makes communication harder and thus productivity lower.</p> <h4 id="hidden-costs">Hidden costs</h4> <p>Especially with more remote teams there will be the need to hire more intermediate people to communicate and portray to the team and the hiring company with the requirements, questions and statuses.</p> <h4 id="knowledge-churn">Knowledge Churn</h4> <p>With consultancies the time to train and retrain is considerable as there normally is a big turnover of staff on the project and outsourced company.</p> <p class="post-photo-narrow"><img src="/img/posts/2025/08/coding-robot-small.jpg" alt="Robot Coding" /></p> <p class="post-photo-license"><em><a href="https://stockcake.com/i/future-coding-robot_2124224_1196273">Photo by Stockcake. Public Domain</a></em></p> <h2 id="comparable-use-of-ai">Comparable use of AI</h2> <p>Now what if an in house development team uses AI agents in their SDLC - how does that compare?</p> <h4 id="like-junior-devs">Like junior devs</h4> <p>Like I discussed in <a href="https://blog.flurdy.com/2025/08/ai-pair-programming.html">Pairing AI Agents</a>, AI in SDLC is like working with junior developers.</p> <h4 id="developers-become-architects-and-managers">Developers become architects and managers</h4> <p>And since it is like delegating to a junior, every developer transforms into their own manager/architect with responsibility to create the requirements and delegate it to the AI agent(s).</p> <h4 id="horizontal-scaled-up-ai-use">Horizontal scaled up AI use</h4> <p>With every developer using AI in such manner is the outcome similar to outsourcing?</p> <h3 id="sdlc-with-ai-benefits">SDLC with AI benefits</h3> <p>Lets compare the pros and cons of outsourcing discussed briefly above.</p> <h4 id="ai-capacity">AI Capacity</h4> <p>If the team goes all in with AI agents the theory and current data indicate they can now scale the horizontally bandwidth by quite a bit. A team can be less developers, so they can have more teams with the same amount of developers.</p> <h4 id="ai-flexibility">AI Flexibility</h4> <p>Whilst the number of engineers does not change, the skills of the AI agents can make the developers much more flexible in terms of programming languages and frameworks.</p> <h4 id="ai-cost">AI Cost</h4> <p>AI agents do cost a lot especially if you go all in. But so does outsourcing. It becomes a calculation of gains v cost.</p> <h4 id="ai-risk">AI Risk</h4> <ul> <li>The employee salaries does not increase.</li> <li>The AI agent cost increases but on a whim can be dialled to 0.</li> <li>Falling behind risk is low as the AI skills are up to date.</li> <li>One risk is not keeping up the developer skills in line with the AI’s skills.</li> </ul> <p class="post-photo-narrow"><img src="/img/posts/2025/08/terminator.jpg" alt="Terminator" /></p> <h3 id="ai-fallacies">AI fallacies</h3> <p>And the negatives of using AI agents in SDLC versus outsourcing:</p> <h4 id="ai-productivity">AI Productivity</h4> <p>The productivity gains could be huge. If managed well on the individual level.</p> <p>YAGNI is vital as with great power comes great responsibility… to not bloat the requirements and features by including everything, and never really finish/deliver.</p> <h4 id="ai-hidden-costs">AI Hidden costs</h4> <p>The AI agent cost is visible. There are no more salary costs.</p> <h4 id="ai-knowledge-churn">AI Knowledge churn</h4> <p>The skills of the AI is permanent, though undoubtely which AI agent is used will constantly be changing in a AI ability race.</p> <p>The skill in using the AI is worth training and focussing on.</p> <p>As well as making sure the skills of the developers don’t wither. They need to supervise and work in conjunction with the AI to avoid AI slop and rotting code.</p> <p class="post-photo-narrow"><img src="/img/posts/2025/08/ai-office-small.jpg" alt="AI Office" /></p> <p class="post-photo-license"><em><a href="https://stockcake.com/i/neon-data-nexus_2678753_1500736">Photo by Stockcake. Public Domain</a></em></p> <h2 id="outsource-or-ai">Outsource or AI?</h2> <p>As you can see above many of the benefits of outsourcing is also prevalent with using AI in SDLC. And many of the negatives are irrelevant or reduced.</p> <p>So should companies just use AI instead? It is not quite as black and white.</p> <h2 id="it-depends">It depends…</h2> <p>As with all things each scenario/company will have different requirements that may make it more desirable or less desirable to use AI over outsourcing.</p> <h3 id="for-many-yes-use-ai-instead">For many, yes use AI instead</h3> <p>For a lot of companies however they should just let the developers go all in with AI and not outsource. Or not outsource their core business skills instead just their fringe/maintenance projects.</p> <h3 id="for-some-still-outsource">For some, still outsource</h3> <p>Some companies they just do not want many developers or have the ability to train theirs, so for them they still need to outsource most of their IT projects.</p> <h3 id="best-of-both-worlds-the-consultancies-use-ai-as-well">Best of both worlds: The consultancies use AI as well</h3> <p>Of course all the consultancies that offer outsourcing will also use AI agents.</p> <p>Their own consultants will learn the skills to delegate to AI agents as well.</p> <p>That means it should require less consultants to fulfil the requirements from the hiring company. So in theory their cost / price should be less…</p> <h3 id="what-about-outsourcing-to-cheaper-countries">What about outsourcing to <em>cheaper</em> countries?</h3> <p>An area that I think will be hit is outsourcing to remote cheaper countries.</p> <p>The need for them will decrease as more companies will be able to either do more internally or hire local consultancies but fewer headcounts so the need to save money will be less of a priority.</p> <p>Time will tell.</p> <h3 id="disclosure">Disclosure</h3> <p>I, the author, Ivar Abrahamsen, have worked as a consultant for a huge multinational consultancy that offered outsourcing, I have been hired in on many projects to a smaller consultancy that do both local mixed with remote outsourcing. As well as a permanent employee of companies that outsourced parts of their projects. Some of the outsourcing have been to India, some near-shore like Czechia, some in the similar timezones in South-Africa and Argentina.</p> <p>I am an independent consultant and available to help your company’s AI journey: <a href="https://eray.uk">eray.uk</a></p>flurdyIs outsourcing development business at risk? By AI agents? Does it replace the reasons people outsource?Pairing AI agents? They’re just like juniors2025-08-02T13:27:00+00:002025-08-02T13:27:00+00:00https://blog.flurdy.com/2025/08/ai-pair-programming<h3 id="can-we-pair-program-with-2-agents-together-is-that-similar-to-pairing-juniors">Can we pair program with 2 agents together, is that similar to pairing juniors?</h3> <p>As I and all of us are getting used to use AI agents in our development flow, I am wondering if we should learn from our experience of the value of pair programming, and with that I mean let AI agents pair with each other?</p> <p>That should get some of the benefits of code quality though less of the knowledge sharing that pairing brings.</p> <p>(Summer of 2025 caveat. Who knows where we are in a year, 5 years or 20…)</p> <h2 id="junior-developers">Junior developers</h2> <p>When using AI agents in your SDLC, with your normal development task, you quickly see the patterns of working with junior developers in a similar fashion.</p> <ul> <li>They can be smart and develop great things by themselves.</li> <li>With a simple task description they may create great solutions.</li> <li>They think of things you did not see and present solutions solving that.</li> <li>They do sometimes see what others have done and keep applying the standard.</li> </ul> <p class="post-photo-narrow post-photo-tall"><img src="/img/posts/2025/08/android-girl-computer-small.png" alt="Android Girl Computer" />]</p> <p class="post-photo-license"><em><a href="https://pixabay.com/illustrations/child-android-computer-girl-young-8803596/">Image by Stephane from Pixabay. Pixabay License</a></em></p> <h4 id="junior-dev-risks">Junior dev risks</h4> <p>But:</p> <ul> <li>They can also go on a crazy train and unnecessary rewrite fundamental parts of the architecture.</li> <li>They make rookie mistakes that more senior developers all did and learnt not to do.</li> <li>They can overcomplicate; and try and do too much instead of focusing.</li> <li>Or they ignore red flags and hyper focus on the tiny tasks ignoring the blatant side effects.</li> <li>They may write extremely bloated code and solution when a DRY terse solution would be much better.</li> <li>They may not be aware of prior art and reinvents the wheel.</li> <li>They do not consider the bigger picture, of how the solution works with other systems and existing solutions.</li> <li>They do not consider the risk and cost on supporting a solution, extending it, training new devs, fixing bugs 5 years in the future etc.</li> </ul> <p>Now seniors (me included) also do these mistakes, but hopefully a lot less after having learnt from early days mistakes.g</p> <h4 id="junior--ai-agent-for-now">Junior = AI agent. For now.</h4> <p>Now it is obvious that AI agents seem to behave like a junior dev, though with time the LLM models may train and train and become more “senior”.</p> <p>But in essence I treat the AI agents as a junior dev that I have to oversee.</p> <p class="post-photo-narrow"><img src="/img/posts/2025/08/surfing-teaching.jpg" alt="Hut" />]</p> <p class="post-photo-license"><em><a href="https://www.pexels.com/photo/photo-of-man-guiding-a-boy-in-surfing-sea-waves-6299967/">Photo by Kampus Production. Creative Commons license</a></em></p> <h2 id="guiding-juniors">Guiding Juniors</h2> <p>You do need junior developers, a constant income of new “blood” is essential. For new ideas, avoiding stagnation and counter natural employee turnovers. And most companies and leaders know how to reduce the risks and side effects of having junior developers.</p> <ul> <li>You have a thorough recruitment process to hire smart, independent, eager-to-learn and responsible employees.</li> <li>You offer training to always expand (and maintain) the developers’ skills and knowledge.</li> <li>You follow up and regularly talk how their process is going and if any changes is needed.</li> </ul> <h4 id="onboarding-and-guidance">Onboarding and Guidance</h4> <p>But the most effecting is having a tried and tested modern onboarding process.</p> <p>This would usually involves pairing up with another developer for a while at the start to ensure the path chosen from the go is the correct one.</p> <p>You teach them the technology, architecture and projects but also the culture and the norms of the company and team.</p> <p>With some guidance the mistakes and the risk of car crashes can be minimised. It is okay to break convention, try something new, just in a sensible manner.</p> <h2 id="pair-programming">Pair programming</h2> <p>Now I have preached many times on the subject:</p> <ul> <li><a href="https://blog.flurdy.com/2017/03/pair-first-pair-always-never-occasionaly">[1] Pair first, pair always, never occasionally</a></li> <li><a href="https://blog.flurdy.com/2015/03/pair-up-now">[2] Pair up now</a></li> <li><a href="https://blog.flurdy.com/2013/11/code-review-with-people-you-dont-like">[3] Pair with people you like and code reviews with people you don’t like</a></li> </ul> <p>In my mind it is essential you keep pairing beyond the initial onboarding.</p> <p>And not just junior with a senior but junior+junior and senior+senior.</p> <p class="post-photo-narrow"><img src="/img/posts/2025/08/human-robot-small.jpg" alt="Human and Robot" />]</p> <p class="post-photo-license"><em><a href="https://cm.taltech.ee/pair-programming-conference-cyprus">Image by Creative Matters</a></em></p> <h2 id="guide-an-ai-agent">Guide an AI agent</h2> <p>It is the same with AI agents.</p> <ul> <li>You guide them with advise</li> <li>You tell them what to do with clear prompts</li> <li>You give them rules to abide by</li> <li>You give them specs to start work on</li> </ul> <h2 id="pair-with-an-ai-agent">Pair with an AI agent</h2> <p>Essentially sometimes it is very much like a top down Senior/Tech-Lead instructing what a junior should do.</p> <p>This depends on what tasks you and the AI are doing. A more hands-off task is a bit more like normal senior/manager to junior delegation of tasks.</p> <p>However when it is an intensive task I think of it as a pair-programming session. Except I am always the navigator/co-pilot and the AI is always the driver/pilot. It is always the navigator that is actually in charge.</p> <h3 id="ai-agent-behaving-like-a-junior">AI Agent behaving like a junior</h3> <p>That is mostly to avoid a lot of the same negative/risks listed at the start for a junior dev.</p> <p>The AI Agent often goes on a completely wrong tangent, or forgets something basic about the SDLC of this project. A hands-on navigator keeps it inline.</p> <p>An experienced navigator also good at catching and knows when it needs to prevent the AI going down a futile deep rabbit hole for a simple solution.</p> <p>This means the navigator has to be on alert and continually aware of each step the AI is changing.</p> <h2 id="pair-2-ai-agents-with-each-other">Pair 2 AI agents with each other.</h2> <p>Now a solution I was thinking to avoid the pitfalls of a junior dev / AI agent maybe the AI should pair up with another AI? To keep it in line?</p> <p>Perhaps a Google Gemini Pro agent can be the navigator for Claude Code Opus agent?</p> <p class="post-photo-narrow"><img src="/img/posts/2025/08/two-robots-work-together-small.jpg" alt="Two robots" />]</p> <p class="post-photo-license"><em><a href="https://www.freepik.com/premium-ai-image/two-futuristic-robots-work-together-laptop-showcasing-potential-ai-workplace_323753155.htm">Designed by Freepik</a></em></p> <h4 id="why-pair-2-agents">Why pair 2 agents?</h4> <p>To try to avoid the obvious negatives of AI agents (and junior developers),</p> <ul> <li>avoiding going down rabbit holes</li> <li>complete rewrites</li> <li>checking if rules was adhered to</li> <li>keeping to conventions</li> </ul> <p>And general common sense that I hope you can prompt an AI to achieve.</p> <h4 id="unnecessary">Unnecessary</h4> <p>People will correctly say that within the LLM there are probably layers and branches of the LLM doing exactly this already. So is the 2 agent pairing redundant?</p> <p>I think if they are 2 different LLMs then they can police each other better.</p> <p>It is worth an experiment. I will try to see how to do it.</p> <h2 id="ai-in-charge">AI in charge</h2> <p>Now, the thing is, when will the AI be the senior delegating tasks to human junior developers?</p>flurdyCan we pair program with 2 agents together, is that similar to pairing juniors?MVP is too late, launch first!2023-06-14T19:00:00+00:002023-06-14T19:00:00+00:00https://blog.flurdy.com/2023/06/mvp-too-late-launch-first<h3 id="launch-first">Launch first</h3> <p>The first thing a project should do is to go live. Yes, live - as in <strong>prod</strong>. Immediately.</p> <h3 id="we-have-not-made-anything-yet"><em>“We have not made anything yet!”</em></h3> <p>That is true, but that should not stop you. You can go live with hardly anything. There are many ways to define a <em>go live / launch</em>.</p> <h3 id="dark-launch">Dark launch</h3> <p>Most of the time it will be a dark launch at the start.</p> <p>Also <em>go live</em> now does not neccesarily go live with the whole intended system. Just a tiny partial launch.</p> <p>Maybe all you go live with is a small listener that mirrors all requests to the current legacy system and then also puts it on a queue nothing listens to yet. The old flow still handles everything. But soon MVP and more will start to act on the replicated requests as well.</p> <p>Maybe it is a tiny app that gets called with real live request but simply redirect to the old system like a <a href="https://en.wikipedia.org/wiki/Facade_pattern">Facade design pattern</a>.</p> <p>Or an unlinked website url, being mostly blank at the start. This does run the risk of being exposed if URL is leaked or sniffed somehow by search engine crawlers.</p> <p>Or your ingress tools can then start to filter 0.001% of traffic to your service etc.</p> <p>Maybe other minor launches that act similarly.</p> <p class="post-photo-narrow"><img src="/img/posts/2023/06/camouflage.jpg" alt="Cheetah camouflage" />]</p> <p class="post-photo-license"><em><a href="https://safarisafricana.com/animal-camouflage/">Photo via Safaris Africana</a></em></p> <h3 id="mvp-is-too-late">MVP is too late</h3> <p>Waiting to launch until when a MVP is ready may be some time. Even a small MVP is a significant delay.</p> <p>Delaying till MVP it basically a mini big bang release.</p> <p>And you loose the benefits of launching even earlier:</p> <p class="post-photo-narrow"><img src="/img/posts/2023/06/missed-train.jpg" alt="Missed Train" />]</p> <p class="post-photo-license"><em><a href="https://pixabay.com/photos/berlin-train-movement-red-4299732/">Photo by spaero from Pixabay. Pixabay License</a></em></p> <h4 id="real-data">Real data</h4> <p>The important part of launching early is to start using <em>real production data</em> and run in the <em>real production environment</em>.</p> <p>Instead of faking maybe unrealistic traffic, or even totally ignorable data, in a substandard environment.</p> <h4 id="real-metrics-and-feedback">Real metrics and feedback</h4> <p>The feedback from metrics and other feedback are from prod usage and more valuable than internal testing feedback.</p> <p>And with early usage data the development can adjust sooner.</p> <h3 id="iterate-on-a-live-product">Iterate on a live product</h3> <p>Better to launch early and iterate on it until you can launch the MVP and beyond. The feedback from real usage is vastly superior to estimated responses.</p> <p class="post-photo-narrow"><img src="/img/posts/2023/06/street-artist.jpg" alt="Street artist" />]</p> <p class="post-photo-license"><em><a href="https://pixabay.com/photos/street-artist-painter-bench-1514087/">Photo by pcdazero from Pixabay. Pixabay License</a></em></p> <h3 id="focus-on-only-deliver-value">Focus on only deliver value</h3> <p>With a live product, only features that actually matter will take priority, and <em>nice-to-haves</em> <em>that-noone-will-ever-use</em> quickly gets culled.</p> <p>Having a live product you will psychologically focus on fixing and implementing the missing exposed parts quickly.</p> <h3 id="keep-going-live">Keep going live</h3> <p>With a minor part live you keep launching more parts live all the time. Eventually the whole delivery is live.</p> <p>And then you keep going.</p> <h3 id="my-real-experiences">My real experiences</h3> <p>I worked on a few projects where we decided to go live very early.</p> <p class="post-photo-narrow"><img src="/img/posts/2023/06/faraday-lecture.jpg" alt="Faraday lecture" />]</p> <p class="post-photo-license"><em><a href="https://en.wikipedia.org/wiki/Royal_Institution_Christmas_Lectures">Public domain</a></em></p> <h4 id="new-retail-rewards">New retail rewards</h4> <p>I worked with a retailer that trialed out a new reward card scheme in a big project.</p> <p>We launched after just 3 weeks. But only in one shop (out of over 1000 stores). And compensating the customers for their inconvenience of using a new incomplete scheme.</p> <p>It worked marvelous. And was gradually expanded to more then eventually all the stores. The early and constant feedback was priceless. We also got early mirror event traffic of the real data flow which was essential in the success.</p> <h4 id="new-insurance-engine">New insurance engine</h4> <p>With a client we launched a new risk pricing calculation engine very quickly. And part of the reason for that success was that we tapped into the existing legacy request events in prod from the start.</p> <p>As we filled in our new services acting on these events the data lake of results quickly matched the legacy system and went beyond.</p> <p>The success was significantly due to the live data feed we got in the first few weeks. That we could compare the old and new continually and deliver value immediately.</p> <h4 id="blog-now">Blog now</h4> <p>On a personal level I posted about <a href="/2019/03/blog-quick-blog-now">Blog quick. Blog now!</a>, where I emphasised the idea of posting blog posts straight away when incomplete.</p> <p>You then quickly want to fill in the rest of your blog post, with even more benefits detailed in that blog post.</p> <h3 id="testing-in-prod">Testing in prod</h3> <p>Part of why I think launching early is great is that I really value the <a href="https://launchdarkly.com/blog/testing-in-production-for-safety-and-sanity/"><em>testing in prod</em></a> approach. With trunk-based-development, feature toggles, no PR gating etc, I think the value of having your project/system in prod quickly is an additional great benefit.</p> <h2 id="tldr">tl;dr</h2> <p>Launch early so you get real data and feedback. Before any features are written. Do not wait for any MVP.</p>flurdyLaunch firstNo what ifs2023-03-29T12:11:00+00:002023-03-29T12:11:00+00:00https://blog.flurdy.com/2023/03/no-what-ifs<h2 id="ban-what-ifs-in-meetings">Ban What-Ifs in meetings</h2> <p><strong>TL;DR</strong></p> <p>Many meetings drags on discussing potential what-if scenarios, don’t waste time on those.</p> <p>Instead when you have agreed on the next step, finish the meeting straight away and focus on that step. Then meet again if necessary afterwards.</p> <p>Less what-if - more focus.</p> <h2 id="what-is-a-what-if">What is a “What If”?</h2> <p>Discussions, planning, estimating a scenario that may happen. Usually followed by further <em>“What-ifs”</em> depending on its output. With responses planned in detail. It is a huge waste of time as most will never happen.</p> <!-- We have all been to meetings that gathers the whole team and more in and the majority of the time is spent on what could possible happen and to plan the response to those in detail. And further what-ifs beyond that. --> <h3 id="not-business-wide-what-ifs">Not business-wide what-ifs</h3> <p>The necessity or futility of corporation-wide business scenario meetings are not the focus of this blog post.</p> <p>This post is more about the smaller frequent team/project/department meetings what-ifs.</p> <p class="post-photo-narrow"><img src="/img/posts/2023/03/TheBYN-Bluebirds-JennyHendershot.jpg" alt="long bird meeting" />]</p> <p class="post-photo-license"><em><a href="https://backyardassist.com/best-bird-bath-de-icer-reviews/">Photo by Jenny Hendershot via Backyard Assist</a></em></p> <h2 id="drags-on">Drags on</h2> <h3 id="meetings-that-drag-on-forever-is-my-main-moan">Meetings that drag on forever is my main moan</h3> <p>Many meetings are a quick team huddle to discuss a particular task/issue. Often a plan to handle this issue is quickly brainstormed and agreed on.</p> <p>But… then most meetings do not end there.</p> <p>As mentioned at the top people start to discuss and plan what if the first solution does not work. And more iterations and decision trees beyond that for the second, third, etc solution. These becomes so speculative and of little value as usually situations change too much for them to carry much value.</p> <p class="post-photo-narrow"><img src="/img/posts/2023/03/hotel_california-480.png" alt="Hotel California" />]</p> <p class="post-photo-license"><em><a href="https://ladreaming.com/guidetola/my-favorite-instagram-spots-in-palm-springs/attachment/hotel-california-palm-springs/">Photo by LA Dreaming</a></em></p> <h3 id="using-allocated-meeting-time">Using allocated meeting time</h3> <p>These meetings that drag on I despise. It seems people try to just to fill the time allocated time instead of letting people leave to crack on. I used to love arguing these possible technicalities for the sake of it, but these days I just see the wasted effort.</p> <h3 id="10-minute-meeting">10 minute meeting</h3> <p><strong>If a meeting can finish after 10 minutes then great.</strong></p> <p>Those are good meetings.</p> <h3 id="selfish">Selfish</h3> <p>Quite frequently these never ending meetings can be due to the meeting organiser having a busy schedule, but often the rest of the attendees do not. Don’t waste their time due to your inability to decline meetings.</p> <h3 id="depressing">Depressing</h3> <p>Not just the wasted time is upsetting, but spending time and effort planning for a future that do not happen can affect moral. People may start to look forward to something they have discussed in detail only to realise later on that it will never happen. Over and over again.</p> <!-- And sometimes the discussions themselves can turn out not to be team building but have the opposite affect if the discussions become confrontational or someone always trumps their preference etc. --> <p class="post-photo-narrow"><a href="https://techcrunch.com/2010/04/23/time-is-money-a-review-of-the-meeting-cost-calculator-and-clock/?guccounter=1"><img src="/img/posts/2023/03/time_money-1280.jpg" alt="Expensive meetings" /></a></p> <p class="post-photo-license"><em>License unknown</em></p> <h2 id="meetings-are-expensive">Meetings are expensive</h2> <p>Meetings themselves are expensive. The more the larger the crowd. People have basically downed tools during the meeting. And interrupts teams, pairs, individuals productivity a lot. So affects for longer than just the meeting allocation.</p> <p>Analyses preparation before, estimation, documentation afterwards, etc are further expense on top of these meeting costs.</p> <h4 id="anecdote">Anecdote</h4> <p>I still remember 15 years ago when I was working for a consultancy where most of the department was body-shopped to clients on-site as individual contractors. Sometimes however everyone was called into the HQ office for some basic info and “team” bonding. In one of these meetings everyone was in a large meeting room.</p> <p>The CEO, or maybe it was the CFO, popped his head through the door. His eyes grew wide, nearly watering, as he saw 30 of his contractors that was not at the clients invoicing. He gasped: <em>“Wow this is an expensive meeting!”</em></p> <h2 id="plans-are-worthless-but-planning-is-everything">Plans Are Worthless, But Planning Is Everything</h2> <p><em>Misquote from Dwight D. Eisenhower</em></p> <p class="post-photo-narrow"><img src="/img/posts/2023/03/construction_workers-800px.jpg" alt="Contruction workers" /></p> <p class="post-photo-license"><em><a href="https://www.designingbuildings.co.uk/wiki/File:Construction_Workers.jpg">Photo by Designing Buildings</a></em></p> <h3 id="is-planning-not-essential">Is planning not essential?</h3> <p>Being unprepared is naïve and irresponsible.</p> <p>But you do not need to predict the whole future, and plan and estimate it in detail all the time.</p> <p>Your architects, analyst, UX researchers, project and product owners, etc will naturally have to have some foresight into the funnel coming up. That is in their job description, but no need to involve everyone in all of these when they are speculative.</p> <h4 id="heads-up">Heads up</h4> <p>Sure, a quickfire of what can happen and what may be the future actions needed is sensible. And two-way communications is invaluable.</p> <p>But drop the details and estimations until it becomes relevant.</p> <!-- #### Woolly roadmap A clear roadmap of what is the business plans helps everyone to understand *why* they are doing the current work. But again, keep it woolly, without detail as that would be guesswork and most likely unfulfilled dreams. --> <h4 id="groom-a-little">Groom a little</h4> <p>I do not have a problem with <em>Grooming</em> your backlog with the team.</p> <p>But not in every meeting. And only a tiny bit in brief sessions.</p> <p>And I have a problem with the need for estimating, see below.</p> <h4 id="optional-inclusion">Optional inclusion</h4> <p>Feel free to include people, inform people of future plans but do no make it mandatory to plan what-ifs for everyone, all the time.</p> <h4 id="optional-departure">Optional departure</h4> <p>Or alternatively let people leave when you enter the futile <em>What-If</em> zone in a meeting?</p> <p class="post-photo-narrow"><img src="/img/posts/2023/03/team_focus.jpg" alt="Team focus" /></p> <p class="post-photo-license"><em><a href="https://pixabay.com/photos/team-huddle-togetherness-connection-1928848/">Photo by Chris Reading from Pixabay. Pixabay License</a></em></p> <h2 id="focus-iteratively">Focus, iteratively</h2> <h3 id="the-next-step-only">The next step, only</h3> <p>When you have agreed on what to do next, just end the meeting and focus on it together straight away.</p> <!-- First just try the agreed initial next step. Properly. Only. --> <p>And then only afterwards look into detail of the following <em>what-ifs</em> once you know the outcome of the initial step.</p> <p>Then plan and focus for the new next step. Only.</p> <h2 id="meetings-are-cheap">Meetings are <em>“cheap”</em></h2> <p>Not the time as explained above if in large long meetings, but organising quick short meetings with just the people concerned should be easy, and if brief relatively cheap.</p> <p>These days a quick ad-hoc Zoom/Teams meeting or Slack/Discourse huddle is so effortless.</p> <h3 id="jitp---just-in-time-plans">JITP - Just-in-time-plans</h3> <p>Only have planning meetings for eventualities that will definitely happen. And you know that only when it is just about to happen.</p> <p>I an not a civil engineer, that may need to plan ahead in detail as physical construction issues can be expensive to undo.</p> <p>But I am a software engineer. We can have meetings and plan <strong>just enough</strong>, act on it, evaluate, then plan just enough again. Iteratively. i.e. “Agile/Lean”.</p> <h2 id="no-estimations">No estimations</h2> <p>These problem stems often from the requirements to estimate everything. Instead we can benefit from the <a href="https://ronjeffries.com/xprog/articles/the-noestimates-movement/">no-estimates</a> convention. You would then have no need to continually plan future work in detail to have <em>guestimates</em>.</p> <p>A lot of time and frustration would be spared.</p> <p>My last few years have been on projects with no-estimation. It has been a lot more productive. And less frustrating.</p> <p class="post-photo-narrow"><img src="/img/posts/2023/03/unimpressed_old_man-480.jpg" alt="Unimpressed man" /></p> <p class="post-photo-license"><em><a href="https://www.pxfuel.com/en/free-photo-odyxz">Photo from Pxfuel. Public Domain</a></em></p> <h2 id="impatient-less-theatre">Impatient. Less theatre</h2> <p>Maybe a lot of these issues is due to that I seem to get more impatient as I get older, not less. With having kids and a lot more experience you would have thought I would be more mellow and understanding.</p> <p>But I am not.</p> <p>I really can not stand all the unnecessary theatre that happens all the time. I have no desire to waste any energy on that.</p> <p><strong>Go JITP! Boo What-if!</strong></p>flurdyBan What-Ifs in meetingsDegoogling2020-08-12T21:12:00+00:002020-08-12T21:12:00+00:00https://blog.flurdy.com/2020/08/degoogling<h2 id="degoogling-myself">Degoogling myself</h2> <p>I am in the process of detaching myself from Google. By using less and less of their services, so that I depend less on Google the company.</p> <p>It is not easy. Google/Alphabet have services and applications that cover nearly everything that I use. And are often the best alternative, sometimes by quiet a large margin.</p> <p>This degoogling process started years ago and is still in progress.</p> <p>And then I also am “Regoogling” myself using new Google services. 🤦</p> <p>(The title is a spin on <a href="https://twitter.com/tornorbye">Tor Norbye</a>’s 2010 tweet when he <a href="https://twitter.com/tornorbye/status/25509797878">cryptically hinted he was joining Google</a>.)</p> <h2 id="why-degoogling">Why degoogling</h2> <p>Risk.</p> <p>Depending on just one company for a lot of my digital life is risky. Suddenly I may not be able to use any of their services or access my data. That could be very problematic.</p> <p class="post-photo-narrow"><a href="https://commons.wikimedia.org/wiki/File:Jenga_distorted.jpg"><img src="/img/posts/2020/08/jenga.jpg" alt="Balancing Jenga" /></a></p> <p class="post-photo-license"><em><a href="https://commons.wikimedia.org/wiki/File:Jenga_distorted.jpg">Photo by Guma89. CC by-sa</a></em></p> <h4 id="google-might-close-down">Google might close down</h4> <p>Google/Alphabet may go bankrupt. Unlikely as they are a very profitable global company, but quick surprise financial disasters have struck big companies before.</p> <h4 id="google-might-get-banned">Google might get banned</h4> <p>The UK or the EU may for some reason ban access to some of Google services. For lack of tax payments, monopoly infringements or political security reasons.</p> <p>Or the US might ban access to European users for some whimsical reason.</p> <p>Something like this might happen but usually with a lot of warning ahead of time and probably only affect a certain service temporarily.</p> <h4 id="google-might-ban-me">Google might ban me</h4> <p>This is the most likely.</p> <p>Many people misuse Google’s services and are rightly banned.</p> <p>Unfortunately as Google services are used by so many millions they automate most of their interactions with their “customers”, including getting banned.</p> <p>This has let to many false positives and innocent people getting banned. Some just temporary, some permanently from one of the services, and some a total and immediate ban from all services for life.</p> <p>And the main problem is that it is Google’s policy to not to tell you specifically what they think you have infringed. This makes it hard to figure out how to you can fix whatever the problem is. Whether is it is an error, a misunderstanding, just a mistake, or nothing wrong at all.</p> <p>If one service ban in error might lead to a total ban, including your email, all your documents, all your photos, analytics, servers etc it is very risky to rely totally on one provider like Google.</p> <p class="post-photo-narrow"><a href="https://commons.wikimedia.org/wiki/File:East_Side_Eviction.jpg"><img src="/img/posts/2020/08/evicted.jpg" alt="Evicted" /></a></p> <h2 id="i-got-banned">I got banned</h2> <p>I had an early taste of the automated false positives as I had an Adwords account closed for no apparent reason which lead me to wake up and realise <a href="https://en.wikipedia.org/wiki/First_they_came_...">they might come for me</a> so I started the formal process of degoogling myself.</p> <h2 id="degoogled-so-far">Degoogled so far</h2> <h4 id="gmail">GMail</h4> <p>Whilst I do host my own mail servers <a href="https://flurdy.com/docs/postfix/">flurdy.com/docs/postfix</a>, for my main accounts they were still hosted with GMail.</p> <p>I have migrated these to <a href="https://fastmail.com">Fastmail</a>. Highly recommended.</p> <h4 id="search">Search</h4> <p>Google search is still the best search engine.</p> <p>But I have for a number of years used <a href="https://duckduckgo.com">DuckDuckGo</a> as my default search provider. But I still often refer to Google for some searches.</p> <h4 id="android">Android</h4> <p>I got so fed up with Android several years ago that I switched to IOS and can’t see myself changing that.</p> <h4 id="blogger">Blogger</h4> <p>I wanted to be more in control of <a href="https://blog.flurdy.com">my blog</a>, and as part of reducing risk by degoogling myself I migrated my blog to a self-hosted <a href="https://jekyllrb.com/">Jekyll</a> instance.</p> <p>I wrote a blog post on <a href="https://blog.flurdy.com/2019/02/migrating-blog-away-from-blogger.html">why I left Blogger</a>, and one on <a href="https://blog.flurdy.com/2019/02/migrate-blog-to-jekyll.html">how to migrate/create a Jekyll based blog</a>.</p> <h4 id="google-drive">Google Drive</h4> <p>I have always used Dropbox and never used Drive as an online storage.</p> <h4 id="chrome">Chrome</h4> <p>5ish years ago I went back to use <a href="https://www.mozilla.org/en-GB/firefox/new/">Firefox</a> as my default browser. With the <a href="https://addons.mozilla.org/en-US/firefox/addon/multi-account-containers/">Multi-Account-Container addon</a> to sandbox Google, Facebook, etc…</p> <h4 id="adwords">Adwords</h4> <p>As mentioned Google closed my Adwords account. And since they specifically ban you from creating another account, I can’t see any way for me to every use Adwords again. Nor recommend it to anyone else.</p> <p class="post-photo-narrow"><a href="https://www.goodfreephotos.com/people/man-dressed-in-white-praticing-tai-chi-on-beasch.jpg.php"><img src="/img/posts/2020/08/taichi.jpg" alt="Tai Chi" /></a></p> <p class="post-photo-license"><em><a href="https://www.goodfreephotos.com/people/man-dressed-in-white-praticing-tai-chi-on-beasch.jpg.php">Photo by Goodfreephotos. Public domain</a></em></p> <h2 id="still-googling">Still googling</h2> <p>For some services I have not found a good enough alternative or I am too lazy to switch away.</p> <h4 id="google-maps">Google Maps</h4> <p>Google Maps is still the best mapping solution. Some alternatives are good and I try to use Apple Maps, OpenStreetMap, TomTom for some parts but still the default is to use Google Maps.</p> <h4 id="google-docs">Google Docs</h4> <p>I have not yet found good enough alternatives to Google Docs for editing documents in a browser. Their spreadsheet and text editor is still the easiest and smoothest to use.</p> <h4 id="adsense">Adsense</h4> <p>Whilst I have dabbled with other ad networks, Adsense is the easiest to use so I have no immediate plans to change that.</p> <h4 id="analytics">Analytics</h4> <p>Google Analytics is easy to use, but if I might try others soon to avoid the snapshot based data issue.</p> <h4 id="google-app">Google App</h4> <p>The IOS search app is actually quite good, both as a search engine but also as a news aggregator.</p> <p class="post-photo-narrow"><a href="https://giphy.com/gifs/reactiongifs-mrw-awkward-13e4OMO2IVZloQ"><img src="/img/posts/2020/08/grampasimpson.gif" alt="Grampa Simpson" /></a></p> <h2 id="regoogling-myself">Regoogling myself</h2> <p>At the same time as I am reducing my exposure to Google I am starting to use some of their other services more.</p> <h4 id="gcp">GCP</h4> <p>I have in the last few years started to use Google’s Computing Platform, especially their Kubernetes based GKE. I have a few clusters on there for my own development and some live services. And might migrate more productions loads to GCP away from AWS and DigitalOcean. Though they have lately started to charge more which is an issue.</p> <h4 id="fitbit">FitBit</h4> <p>Not sure if the acquisition is approved/complete yet but I like my FitBit Charge. Though if ever Apple releases a watch that is not square and dorky like their current ones then I could be tempted. I blame Pebble for making me wear watches again.</p> <h4 id="nest">Nest</h4> <p>I love my Google Nest Mini so much we now have 3 of them. And I can see myself buying other Nest products such as the doorbell.</p> <h4 id="google-assistant">Google Assistant</h4> <p>Though not foolproof I am impressed by Google Assistant. And much prefer to use it instead of Alexa and Siri.</p> <h4 id="google-photos">Google Photos</h4> <p>I have some very old albums from the Picasa days. But I avoided their Photos service like a plague when Google forced us to use Google Plus which I refused.</p> <p>Since Google Plus is no more and Google Photos is now a standalone service it is again an alternative especially as many people I know highly recommend it.</p> <p>So I plan to start migrating my albums to it. Though still duplicated in iCloud Photos and Dropbox.</p> <h2 id="to-google-or-not-to-google">To Google or not to Google</h2> <p>Well both.</p> <p>I think I have reduced my exposure and risk. And increased back up options. But I know I still am and will be heavily reliant on Google’s services as they make great products.</p> <p class="post-photo-narrow"><a href="https://thefutureofai.blogspot.com/2018/05/google-dont-be-evil.html"><img src="/img/posts/2020/08/dontbeevil.png" alt="Don't be evil" /></a></p>flurdyDegoogling myselfMicroservices are cattle not pets2019-10-09T16:08:00+00:002019-10-09T16:08:00+00:00https://blog.flurdy.com/2019/10/microservices-are-cattle-not-pets<h2 id="shed-no-tears-when-quickly-culling-services">Shed no tears when quickly culling services.</h2> <p>I tweeted a few years ago that microservices are cattle. It is an advice I give to developers to not be precious about any microservices that they may have been involved with.</p> <blockquote class="twitter-tweet"><p lang="en" dir="ltr">With clouds servers are cattle not pets. With microservices the services are cattle not pets. Shed no tears when quickly culling services</p>&mdash; Ivar Abrahamsen (@flurdy) <a href="https://twitter.com/flurdy/status/721490238443495424?ref_src=twsrc%5Etfw">April 17, 2016</a></blockquote> <script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> <h2 id="cattle-or-pets">Cattle or pets</h2> <blockquote> <p>Servers are cattle, not pets - Randy Bias</p> </blockquote> <p>It is thought the server cattle analogy was <a href="https://www.slideshare.net/randybias/architectures-for-open-and-scalable-clouds">first mentioned around 2012 by Randy Bias</a>. Whom in turn was influenced by Bill Baker’s “<em>Scale out not up</em>”. Randy <a href="https://cloudscaling.com/blog/cloud-computing/the-history-of-pets-vs-cattle/">elaborated in 2016 a bit about the analogy’s background and history</a>.</p> <h4 id="pet-servers">Pet servers</h4> <p>In essence in earlier times with co-located servers you knew each of the servers’ names, maybe even had a charming naming policy, and you would be upset if something happened to them. You maintained and fixed each server over a long period of time.</p> <p>I.e. a pet.</p> <p><a href="https://www.flickr.com/photos/21265997@N08/5148597478"><img src="/img/posts/2019/10/big-dog-kitten.jpg" alt="Big dog and kitten" /></a></p> <p class="post-photo-license"><em><a href="https://www.flickr.com/photos/21265997@N08/5148597478">Photo by Katlene Niven. CC by</a></em></p> <h4 id="cattle-servers">Cattle servers</h4> <p>However if in a modern cloud virtual environment where you scale out horizontally, massively, you would not really care if <em>mail-server-238</em> had an issue. You would simple mercy kill it and instead add a few more instance nodes in seconds. And not be upset by it at all, it is just work.</p> <p>I.e. cattle.</p> <p>And this would possibly be automatic so to you it may merely be a minor datapoint in your monthly cloud usage report.</p> <p><a href="https://www.rtl.fr/actu/debats-societe/chine-une-ferme-de-100-000-vaches-pollue-les-nappes-phreatiques-7786462404"><img src="/img/posts/2019/10/cow-factory.jpg" alt="Cow factory" /></a></p> <p class="post-photo-license"><em><a href="https://www.rtl.fr/actu/debats-societe/chine-une-ferme-de-100-000-vaches-pollue-les-nappes-phreatiques-7786462404">Photo by AFP</a></em></p> <h2 id="monolithic-pets">Monolithic pets</h2> <p>As with servers, larges applications and services, often described as monoliths, are treated as pets.</p> <p>Teams defend it and tinker with it for years. They polish it, they care for it and do not ever want to give it up. They extend it, morph it, modernise it, and add more and more features to it. They fix bugs, they even acknowledge problems with and work around it so the application stays alive.</p> <p>I.e. a pet.</p> <p><a href="https://commons.wikimedia.org/wiki/File:A_female_British_Shorthair_cat_on_a_brown_sofa_resting.jpg"><img src="/img/posts/2019/10/sofa-cat.jpg" alt="Sofa cat" /></a></p> <p class="post-photo-license"><em><a href="https://commons.wikimedia.org/wiki/File:A_female_British_Shorthair_cat_on_a_brown_sofa_resting.jpg">Photo by George E. Koronaios. CC by-sa</a></em></p> <h3 id="precious-microservices">Precious microservices</h3> <p>Unfortunately that attitude also transcends to applications in a microservices architecture. Even more if they are not pure microservices but more mini-monoliths.</p> <p>But developers still have affinity to what they create, so they may treat microservices as pets as well.</p> <p>Don’t.</p> <p><a href="https://www.flickr.com/photos/rofanator/5751217677"><img src="/img/posts/2019/10/meerkat-protect.jpg" alt="Meerkat defend young" /></a></p> <p class="post-photo-license"><em><a href="https://www.flickr.com/photos/rofanator/5751217677">Photo by Jon Pinder. CC by-nc-nd</a></em></p> <h2 id="microservices-are-cattle">Microservices are cattle</h2> <p>Treat microservices as cattle. Microservices are meant to be small, easily rewritable and replaceable. And therefor easily deletable.</p> <p>Do not try to resist deleting them because you put a lot of effort into it. Do not be precious about tiny bits of code. Be cold.</p> <p>If not needed, do not shed a tear. Cull it, move on and iterate onwards.</p> <h4 id="be-cold-to-others">Be cold to others</h4> <p>And treat code and microservices written by others with the same ruthlessness. Sure, talk and ask others before deleting it, but do not save any gunpowder. Kill any services that are superfluous.</p> <p>And if others delete a service you worked on a lot, be magnanimous and supportive.</p> <p><a href="https://thomaswdinsmore.com/2018/12/04/ai-ml-roundup-november-2018/"><img src="/img/posts/2019/10/cattle_round_up.jpg" alt="Cattle round up" /></a></p> <p class="post-photo-license"><em>License unknown</em></p> <h4 id="not-clones">Not clones</h4> <p>A difference however between microservices and cloud based servers is that servers are numerous horizontal clones, whilst microservices are numerous but individual services.</p> <p>So there is not a <em>order-48</em> service, but perhaps a <em>receive-order</em>, <em>retrieve-order</em>, <em>cancel_order</em> service or however you split and define, and name your services and context.</p> <p><a href="https://www.wired.co.uk/article/human-cloning-technology-dolly-sheep"><img src="/img/posts/2019/10/Nottingham-Dollies.jpg" alt="Clone sheep" /></a></p> <p class="post-photo-license"><em><a href="https://www.wired.co.uk/article/human-cloning-technology-dolly-sheep">Photo by University of Nottingham</a></em></p> <h4 id="dont-morph-kill--replace">Don’t morph, kill &amp; replace</h4> <p>If suddenly your team realised that they no longer need to get an order directly but they instead need to search for orders, don’t hesitate to delete <em>retrieve_order</em> and launch a new <em>search-for-orders</em> service.</p> <p>No need to make it temporarily do two things, and 2 years down the line, after lots of iterations, <em>retrieve_order</em> now also makes coffee.</p> <p>And no need to keep <em>retrieve_order</em> around just in case. Sure, the source code may exist as e.g. an archived Github project for instance, but get rid of all infrastructure immediately.</p> <h2 id="great-herder--gardner">Great herder &amp; gardner</h2> <p>If you keep trimming, deleting and keeping only the bare essential microservices, your project, architecture and infrastructure has a much better chance of being a solid, evolving and successful product for years.</p> <p><a href="https://commons.wikimedia.org/wiki/File:Ayrshire_cow.jpg"><img src="/img/posts/2019/10/Ayrshire_cow.jpg" alt="Ayrshire cow" /></a></p> <p class="post-photo-license"><em><a href="https://commons.wikimedia.org/wiki/File:Ayrshire_cow.jpg">Photo by Malcolm Morley. Public domain</a></em></p>flurdyShed no tears when quickly culling services.All the eggs in one basket2019-06-18T01:01:00+00:002019-06-18T01:01:00+00:00https://blog.flurdy.com/2019/06/all-eggs-in-one-cloud-basket<p>Many companies have all or most of their infrastructure with just one cloud provider. That does have some benefits, but also some drawbacks and serious risks.</p> <ul> <li>What are these benefits of using only one cloud provider?</li> <li>What are the reasons people end up with just one cloud?</li> <li>What are the risks and drawbacks of putting all your eggs in one cloud basket?</li> <li>And what are the alternatives and steps to reduce the risk?</li> </ul> <h2 id="all-in-the-cloud">All in the cloud</h2> <p>Since <a href="https://aws.amazon.com">Amazon’s AWS</a> was launched, followed by <a href="https://cloud.google.com">Google’s GCP</a>, <a href="https://digitalocean.com">Digital Ocean</a>, <a href="https://azure.microsoft.com">Microsoft’s Azure</a>, <a href="https://heroku.com">Heroku</a> and many, many more, the term <em>cloud</em> has become a common well understood term for hosting infrastructure.</p> <p>And from initially being a bit hesitant and just trying it out many companies have now moved most of their infrastructure to <em>the cloud</em>. Especially younger smaller companies with less legacy datacentre usage.</p> <p>Starting a new company or project using co-location datacentre servers just does not make sense any more unless you are on a real shoestring budget where the flexibility and speed of a cloud solution is not possible.</p> <h2 id="stay-with-the-one-cloud">Stay with the one cloud</h2> <p>AWS was the first real cloud provider. There was others before depending on how you define <em>cloud</em> but none came with the size, marketing, and total <a href="https://en.wikipedia.org/wiki/Infrastructure_as_a_service">IAAS</a> solution of AWS. So AWS was the first provider most people dabbled with. And by now 10+ years later it is the giant in this marketplace with as many customers as all the others combined. And with a huge range of features to confuse the best of us.</p> <p>And since companies was already on AWS many have not really looked further at other cloud providers.</p> <p>The same happens with newer companies and projects that perhaps choose other big providers such as GCP or Azure. Once they are on and it works they do not look further as their chosen cloud provider does everything they need.</p> <p>If you only use one provider your employees can specialise and enhance their skills of its specific features.</p> <h2 id="cloud-lock-in">Cloud lock-in</h2> <p>The cloud providers also encourage people not to look elsewhere. As mentioned they provide a lot of IAAS features and continually add more so your requirements are mostly met on the one platform.</p> <p>They provider services that are unique to their platform. Such as <a href="https://aws.amazon.com/lambda">AWS’s Lamda</a> or <a href="https://cloud.google.com/functions/">GCP’s Cloud Functions</a> Or database solutions and data that are not easily transferrable such as <a href="https://aws.amazon.com/dynamodb/">Dynamo</a> or <a href="https://cloud.google.com/spanner/">Cloud Spanner</a>.</p> <p>But mostly they try to spread as wide as possible and ingrate as deep as possible into your whole organisation and procedures so that a migration elsewhere will be very painful and just not feasible.</p> <p><a href="https://en.wikipedia.org/wiki/File:Jean_Baptiste_Creuze_Broken_Eggs.JPG"><img src="/img/posts/2019/06/baptiste-broken-eggs-small.jpg" alt="Broken eggs" /></a></p> <p><em><a href="https://en.wikipedia.org/wiki/File:Jean_Baptiste_Creuze_Broken_Eggs.JPG">Painting by Jean Baptiste. Public Domain</a></em></p> <h1 id="smashed-eggs">Smashed eggs</h1> <p>Having all your eggs in one basket is risky.</p> <p>If all your infrastructure is in the cloud and with just one provider you run the risk of several things crippling your project or whole company.</p> <h2 id="runaway-usage-equals-costs">Runaway usage equals costs</h2> <p>Setting up and using a cloud service is fast, easy and very contagious.</p> <p>The old way of rack based bare metal co-location flow of ordering a new server then setting it up was time consuming, very time consuming, perhaps days if not weeks. A cloud instance is matter of minutes or less.</p> <p>Virtual server solutions, e.g. Zen, VMWare, made this process quicker, but still painful and people were frugal with how often they created a new server.</p> <p>With cloud solutions they are not, and they expand to use more and more storage, databases, queues, and more that are metred costs. When this infects your whole business the usage continues to escalate enormously, control and oversight evaporates and the costs rocket.</p> <h2 id="downtime">Downtime</h2> <p>A big risk that bleeds money is downtime. When your service is offline you loose money. A lot of money.</p> <p>And big cloud providers do go down, taking your services with them offline.</p> <p>AWS has had issues many times, GCP has been offline, and Azure is also known for significant downtime.</p> <p>Some times it is not just a partial outage nor limited to a region, sometimes everything goes offline. For example <a href="https://techcrunch.com/2019/06/02/google-cloud-is-down-affecting-numerous-applications-and-services/">GCP’s recent massive outage</a> that took also down Google’s own Youtube, GMail and more.</p> <p>And when they are down there is not much you can do than wait. Hopefully they are good at communicating the status.</p> <h2 id="automated-suspensions-and-collateral-damage">Automated suspensions and collateral damage</h2> <p>The large cloud providers due to their scale has mostly automated a lot of their procedures. Including detection of illegal activity, abuse reporting and support. And at best have staff on low pay whom have to process a huge amount of tickets by spending as little time on per ticket as possible.</p> <p>I have been scared by numerous reports of random accounts being frozen and service taken offline for no apparent reason. Or tiny misinterpretations that would have been corrected have they been able to talk to a human.</p> <p>But I was suffering from <a href="https://encyclopedia.ushmm.org/content/en/article/martin-niemoeller-first-they-came-for-the-socialists">Niemöller’s quote</a>: “<em>First they came for the socialists, and I did not speak out — because I was not a socialist…</em>” I did not think I was doing anything wrong so this did not involve me.</p> <p>Until last year, when my own <a href="https://ads.google.com">Adwords</a> account was suspended then banned for no apparent reason with no recourse. Thankfully they did not suspend any of my other accounts or use of other Google services as that would affect not just me badly but my family as well.</p> <p>Google have done that to <a href="https://twitter.com/search?q=google%20account%20suspended">others</a>, e.g. suspended all accounts linked and just suspected of being linked, including business and personal accounts. And total ban ie deleting <a href="https://slate.com/technology/2013/04/life-without-google-when-my-account-was-suspended-i-felt-like-id-been-dumped.html">docs, email, photos, Android profiles</a> for all these accounts. They do state that they may do so in their policies.</p> <p>Recently Digital Ocean had some bad PR when one their automated abuse script <a href="https://twitter.com/w3Nicolas/status/1134529316904153089">banned a persons account effectively taking his whole company offline</a>. What made it worse he no longer could access his database backups as they were all hosted with DO. So his company was effectively dead. Thankfully he had enough social media clout to reach one of DO’s founders whom rectified the problem. Most people don’t have that reach.</p> <p>These stories have made me very wary of ever relying on one provider again.</p> <p><a href="https://www.flickr.com/photos/artembali/44927194462"><img src="/img/posts/2019/06/battery-chicken-small.jpg" alt="Battery chicken" /></a></p> <p><em><a href="https://www.flickr.com/photos/artembali/44927194462">Photo by Artem Beliaikin. Public domain</a></em></p> <h1 id="better-egg-distribution">Better egg distribution</h1> <p>So what steps can we take to minimise the risk of depending on a single cloud provider?</p> <h2 id="cost-monitoring">Cost monitoring</h2> <p>You should keep a close look on how much your company is spending on a cloud. And if steps need to be taken to reduce it or just slow it down, or detect any anomalies.</p> <p>The providers offer monitoring, e.g. <a href="https://aws.amazon.com/cloudwatch/">CloudWatch</a> and <a href="https://cloud.google.com/monitoring/">StackDriver</a>.</p> <p>And there is a vast range of 3rd party tools to monitor usage, threats, errors and billing costs across several cloud providers. Such as <a href="https://www.dynatrace.com">Dynatrace</a>, <a href="https://looker.com">Looker</a>, <a href="https://www.datadoghq.com">Datadog</a>, <a href="https://newrelic.com">NewRelic</a>, <a href="https://www.cloudability.com">Cloudability</a>, <a href="https://www.alertlogic.com">AlertLogic</a>, <a href="https://www.cloudhealthtech.com">CloudHealth</a> etc.</p> <p>Though to me they all seems one tiny AWS feature announcement away from going out of business.</p> <p>But its a lucrative marked and <a href="https://techcrunch.com/2019/06/06/google-to-acquire-analytics-startup-looker-for-2-6-billion/">Looker was just acquired by Google</a>.</p> <h2 id="availability-zones-and-regions">Availability zones and regions</h2> <p>Most of the cloud provider downtimes are due to a single datacentre of theirs going offline for some reason. Power cut, builders accidentally digging through a cable outside, etc. So only customers using that single datacentre are affected.</p> <p>So the cloud providers all offer availability zones and multiple regions to mitigate this risk.</p> <p>Availability zones are multiple datacentres in the same city but not the same building. Which reduces the risk of a physical problem at a single building.</p> <p>Regions are datacentre locations spread across the globe. With possibly its own multiple availability zones per site. This reduces the risk of a larger city wide problem.</p> <p>If you modify and design your applications and infrastructure to spread horizontally across the zones and/or regions, you can survive when a providers single datacentre goes down.</p> <p>This horizontally scalable design costs a lot architecture and development time and runtime resources so there is a clear overhead. But you can sleep more easily.</p> <p>However as mentioned if there is a problem with some specific cloud features it may take down all the zones and global regions of that provider. They are rare but they do happen.</p> <h2 id="many-different-baskets-hybrid">Many different baskets, hybrid</h2> <p>Of course the obvious solution to one basket is many baskets. Use several cloud providers, and your own hosted solutions.</p> <p>This is a <em>hybrid</em> solution, and many providers and 3rd parties offering solutions for this. AWS offer an integration with VMWare for an on-premises solution, <a href="https://aws.amazon.com/outposts/">Outpost</a>, that integrates with AWS. Google is offering several tools to manage Hybrid solutions, e.g. (<a href="https://cloud.google.com/anthos/">Anthos</a>). But I am suspicious of these, I am not sure what the motives are for their offering, but I am sure it is to drive more use of their own cloud.</p> <p>Some 3rd parties offer a layer on top that is supposed to <em>smoothly</em> uses different cloud providers underneath. That may work, but I am also not sold on the idea of introducing yet another component. Those I have used are OK but just added more ambiguity and less flexibility.</p> <p>You can architect your applications yourself to cater to work across multiple cloud providers. This is the best solution if a full hybrid solution is desired.</p> <p>You really have to consider eventual consistence and temporary separated and then rejoined clusters. My take is to only do this if you have a big budget, a mature product, and very skilled architecture and ops teams. Most of the time this is a complete overkill.</p> <h2 id="independent-technology">Independent technology</h2> <p>My main opinion on hardened cloud usage is to use independent technology. Use technology stacks that you can use on any cloud provider. And minimise proprietary technology a cloud provider supplies.</p> <p>For instance using <a href="https://kubernetes.io">Kubernetes</a> lets you deploy your solution to many providers. You can run it across their instances, or use their managed services such as <a href="https://aws.amazon.com/eks/">EKS</a>, <a href="https://cloud.google.com/kubernetes-engine/">GKE</a>, <a href="https://azure.microsoft.com/en-us/services/kubernetes-service/">AKS</a> and with <a href="https://www.digitalocean.com/products/kubernetes/">DigitalOcean</a>. Though Kubernetes is a behemoth and not right for everyone.</p> <p>For databases use <a href="https://www.postgresql.org">Postgres</a>, <a href="https://redis.io">Redis</a>, <a href="http://cassandra.apache.org">Cassandra</a>, <a href="https://www.cockroachlabs.com">CockroachDB</a> and others that you can use on any provider and does not tie your data to a specific supplier.</p> <p>Use <a href="https://www.elastic.co/elk-stack">EKL</a> logging so that it is ambiguous if you use AWS or GCP or others.</p> <p>There are also many generic serverless functions solutions. <a href="https://serverless.com/">Serverless</a>, <a href="https://fission.io/">Fission</a>, <a href="https://www.openfaas.com/">OpenFAAS</a>, and I am sure many more to come. I have not tested them well enough to know if they are mature enough yet.</p> <p>To avoid reinventing the wheel, and use the cloud providers automatic scaling ability, I understand people choosing the cloud providers databases and streaming solutions instead of setting up and maintaining all that infrastructure yourself. My projects have often relied on DynamoDB or <a href="https://aws.amazon.com/rds/">AWS’s RDS</a>, and I would consider <a href="https://aws.amazon.com/msk/">AWS’s Kafka service</a> in future projects. So my suggestion is not 100% gospel, just be pragmatic and responsible.</p> <h2 id="back-up-for-total-dataloss">Back up for total dataloss</h2> <p>Always have full back up of your data, and fully restore tested, that is not only stored on your cloud provider.</p> <p>It does not have to be real time slave of the data but a fairly recent full back up. So that in case of total dataloss that may happen if your account is hacked, your account is suspended, or other issues, you can still restore your database to a fairly recent state.</p> <p>That way your company and project is not dead, just very set back.</p> <p><a href="https://pixabay.com/photos/fried-buffet-gastronomy-hotel-1789962/"><img src="/img/posts/2019/06/fried-egg-small.jpg" alt="Fried egg buffet" /></a></p> <p><em><a href="https://pixabay.com/users/darf_nicht_mehr_hochladen-2998623/">Photo by darf_nicht_mehr_hochladen from Pixabay. Public Domain</a></em></p> <h1 id="my-fried-eggs">My fried eggs</h1> <p>So what do I actually suggest for companies cloud strategy?</p> <p>Plan, plan, try, plan, plan.</p> <ul> <li>Be aware of the risks of relying on one provider</li> <li>Plan for sudden migration</li> <li>Implement full back up procedures</li> <li>Test out other providers</li> <li>Use mostly independent technologies</li> </ul> <p>Essentially plan and document a migration. But you don’t always have to implement a full hybrid solution as that can be expensive.</p> <p>But do try out other providers. Perhaps run small quite independent applications on GCP if your main stack is on AWS for example. That way the ramp up and teething problems of using another provider can be dealt with already if a migration is needed.</p> <p>Do trial run migration if you want to wash out any possible hidden gotchas on your reliance of the current provider.</p> <p>And run as much of an independent technology stack as possible.</p> <p>You can and perhaps should use their very scalable database and streaming solutions for any high throughput parts of your architecture. But do back it up and have tested migration plan to another solution in place.</p> <p>Feel free to try out Lamda and similar services, but try to limit its core importance. I.e. can it be replaced quickly.</p> <h2 id="budget-cloud-independence">Budget cloud independence</h2> <p>An important step often missed by plans to move to a cloud provider is to include a budget to move away from it.</p> <p>I know it sounds like a hard sell to your CTO that you are already thinking of moving away from the shiny chosen cloud provider, but it is essential in a costing plan of a bid/project that you will have to consider moving away from it as well.</p> <p>But I am softening up on this requirement to instead include budgets to plan a migration and test other providers, and not budget for a full migration.</p> <p><a href="https://www.flickr.com/photos/53344659@N05/4978438263"><img src="/img/posts/2019/06/egg-chick-small.jpg" alt="Chick and eggs" /></a></p> <p><em><a href="https://www.flickr.com/photos/53344659@N05/4978438263">Photo by sheilapic76. CC-by</a></em></p> <h1 id="any-basket-is-better-than-broken-eggs">Any basket is better than broken eggs</h1> <p>Should all companies running on only one cloud provider follow my advice?</p> <p>No, for most putting all eggs in one basket is calculated risk worth taking.</p> <p>After all, <em>A</em> cloud provider is better than none.</p> <p>And remember the big providers are much better at reliability, fault tolerance and networking than you. Yes, a lot better.</p> <p>Just please consider my suggestion of planning and testing alternatives if you can afford it. Or rather can you afford not to?</p>flurdyMany companies have all or most of their infrastructure with just one cloud provider. That does have some benefits, but also some drawbacks and serious risks.To PAAS or not to PAAS2019-04-07T21:22:00+00:002019-04-07T21:22:00+00:00https://blog.flurdy.com/2019/04/to-paas-or-not-to-paas<h2 id="when-is-a-paas-the-right-tool">When is a PAAS the right tool?</h2> <ul> <li>Who should use it? Who should not?</li> <li>Who can afford it? For whom is it too expensive?</li> <li>Who will benefit? Where will it be restrictive?</li> <li>and When?</li> </ul> <p>I have read, heard and seen many tales and blog posts on how to move away from a PAAS. But those are just information on how to and not why. And they were suitable for that organisation for their requirements at that specific time, not always.</p> <p>I hope to clarify that a PAAS is great for many people/companies. And sometimes not.</p> <h2 id="what-is-a-paas">What is a PAAS</h2> <p><a href="https://en.wikipedia.org/wiki/Platform_as_a_service">PAAS = Platform As A Service</a>.</p> <p>It is a service where you deploy and host your application without having to consider any infrastructure.</p> <ul> <li>You do not need to think about creating or setting up a server instance(s).</li> <li>You do not need to consider any storage or networking.</li> <li>You do not need to worry about OS updates and security patches.</li> <li>You do not need to configure load balancing.</li> <li>And much much more.</li> </ul> <p>Well known PAASes are <a href="http://heroku.com">Heroku</a>, <a href="https://cloud.google.com/appengine/">AppEngine</a> and <a href="https://aws.amazon.com/lightsail/">AWS Lightsail</a>. As well as many more specific tailored towards a niche, for example programming languages such as <a href="https://www.ruby-lang.org/en/">Ruby</a>, etc.</p> <p>Unlike a vanilla cloud offering i.e. an IAAS, <a href="https://aws.amazon.com/lightsail/">Infrastructure As A Service</a>, where you do have to consider all these steps yourself. Well known IAASes are <a href="https://aws.amazon.com/ec2/">AWS EC2</a>, <a href="https://cloud.google.com">GCP</a> and <a href="https://azure.microsoft.com">Azure</a>.</p> <p>There are also many ways you can host a PAAS yourself on your own IAAS or bare metal servers. Such as <a href="https://www.cloudfoundry.org/">Cloud Foundry</a>, <a href="https://www.openshift.com/">OpenShift</a> or <a href="http://dokku.viewdocs.io/dokku/">Dokku</a>.</p> <h2 id="why-not-paas">Why not PAAS</h2> <p>Why are people complaining of PAAS offerings?</p> <p>Some complaints are valid, many invalid.</p> <ul> <li>Cost</li> <li>Inflexibility</li> <li>Ownership</li> </ul> <p>The lack of flexibility is what developers and operations would moan about PAAS. People like to tinker and be in control. Connect this app to that database. Triple the memory on that instance, scale that application to 4 nodes, bundle these three apps on one instance, restrict the firewall on these etc.</p> <p>The cost is what whomever pays the bill would moan about the most. If handled incorrectly a PAAS can be costly, especially if naïvely compared to pure hosting costs of an IAAS.</p> <p>Ownership. People may think with the inflexibility and black-box nature of a PAAS, that they don’t know what is going on. Who can see my company’s data, my users’ requests? Is there a legal risk?</p> <h2 id="paas-is-just-hosting-right">PAAS is just hosting, right?</h2> <p>Often people think of PAAS as a straight swap of the instances in a standard cloud offering (IAAS). And compare the costs and flexibility of that only. But a PAAS is so much more, and that is why it costs so much more <em>per instance</em>.</p> <p>A PAAS offers logging, metrics, backup, load balancing, redundancy, roll-back, deployment, scaling, patching, etc, as part of the one price offering.</p> <p>It offers years of experience, tweaking, to make the services that provide all that to work as smooth as possible, all the time, for everyone.</p> <p>And when there is a network issue they take care of it immediately. Any hardware issue they take care of it. Any security patches - they apply it for you. Most of the time you wont even notice as they bring up another horizontal node of your application and replace your current one without any downtime.</p> <p>And for instance in the case of Heroku they offer <a href="https://elements.heroku.com/addons/">lots of add-ons</a> that is mostly just one click or one CLI command away to be set up and working straight away with your applications.</p> <h2 id="time-is-money">Time is money</h2> <p>With a self hosted IAAS solution on a cloud provider, you have to pay for all the time, i.e. salaries, for people to investigate, set up, fail, retry, manage, maintain, upgrade and more of all your server instances and applications. Every time. All the time.</p> <p>That is expensive.</p> <p>And to be on-call when it all falls over in the middle of the night.</p> <h2 id="is-paas-expensive">Is PAAS expensive?</h2> <!-- ( *Ignoring the free and $7 per month price bands that for example Heroku has, as they are more suitable for personal projects and prototypes.* ) --> <p>For businesses prices at for example Heroku start at $25 per month.</p> <p>That is cheap.</p> <p>Some will counter that you can get a nano instance at AWS for a month for a lot less. But they forget the above mentioned, PAAS is not just hosting.</p> <p>For a company $25 per month is nothing. The more powerful options at $50 or even $250 for a real business is still nothing, even if horizontally scaled to multiple nodes, if the applications are important to the company.</p> <p>Remember you are comparing it to paying for more salaries in a custom infrastructure. Or divert focus away from other tasks for your Operations staff.</p> <p>Remember $25 cost would comparably pay for perhaps 20 minutes or less for an Ops contractor when comparing hourly contractor rates if in e.g. London.</p> <p>So no, PAAS is cheap, not expensive.</p> <p>However once you scale to <em>a lot</em> of apps and nodes, and a lot of traffic on AppEngine for example, PAAS is then no longer the most economical solution. Though at that scale, nothing is cheap.</p> <h2 id="premature-optimisation">Premature optimisation</h2> <p>One thing a PAAS is good for is to prove whether a project/application need heavy infrastructure investment before you commit to it.</p> <p>Try lots of ideas, pivots, applications in a live environment before focussing on one that will succeed.</p> <p>Prove that it is useful and will be worth the return of investment before spending time and money into salaries, tools and IAAS to set up an infrastructure that scales for a big success.</p> <p>Validate the idea, not infrastructure that may not be needed.</p> <h2 id="one-size-fits-no-one">One size fits no-one</h2> <p>The requirements, time and money available is very different depending on your situation:</p> <h4 id="startup-">Startup <i class="icon ion-md-checkmark-circle glyp-green"></i></h4> <p>A Startup is often a good candidate for a PAAS project.</p> <p>A Startup may want to focus on validating the idea, avoiding the above mention premature optimisation. Often they have limited budget and need to focus on a few areas only, of which infrastructure probably is not yet included.</p> <p>A Startup is often time poor, needing to get something launched before anyone else do the same.</p> <p>If the product starts to make profit then they can start to diversify the PAAS into custom infrastructure architecture(s).</p> <h4 id="unicorn-startup-">Unicorn Startup <i class="icon ion-md-close-circle glyp-red"></i></h4> <p>A Startup with big investments and in the process of massively scaling its staff may be able to hire a boat load of Operations employees to create an elaborate architecture and infrastructure.</p> <h4 id="no-opsdevops-">No Ops/DevOps <i class="icon ion-md-checkmark-circle glyp-green"></i></h4> <p>If you are a small company that do not want to hire Operations staff then having your application / website on a PAAS makes sense, they do the majority of the Ops for you.</p> <h4 id="cheap-labour-cost--">Cheap labour cost <i class="icon ion-md-close-circle glyp-red"></i></h4> <p>In a cheaper labour location your company may see benefits in paying salaries to set up and maintain a custom infrastructure, as a PAAS is then probably not an economical choice.</p> <h4 id="new-greenfield-project-">New greenfield project <i class="icon ion-md-close-circle glyp-red"></i></h4> <p>A new greenfield project in a larger organisation often have both the funding and time to go full wack on custom infrastructure.</p> <h4 id="prototyping-">Prototyping <i class="icon ion-md-checkmark-circle glyp-green"></i></h4> <p>Prototyping is an ideal PAAS candidate.</p> <p>For some it is very useful as just a simple UX scaffold to demonstrate a project idea. For others it would make sense as a way of iterating over a full version but still not properly launched.</p> <h4 id="live-asap-">Live ASAP <i class="icon ion-md-checkmark-circle glyp-green"></i></h4> <p>If you need the get a live version straight away, it may make sense to deploy it to a PAAS first. Then iterate on a more elaborate custom infrastructure whilst you already at the same have a live site.</p> <h4 id="monolith-">Monolith <i class="icon ion-md-checkmark-circle glyp-green"></i></h4> <p>If the app is a fairly self contained monolithic application a PAAS can be a good option.</p> <h4 id="microservices-">Microservices <i class="icon ion-md-close-circle glyp-red"></i></h4> <p>An ant farm myriad of a <a href="https://en.wikipedia.org/wiki/Microservices">microservices architecture</a> would be a bad choice for a PAAS.</p> <h4 id="support-tools-">Support tools <i class="icon ion-md-checkmark-circle glyp-green"></i></h4> <p>Simple internal tools may also be a good PAAS candidate. They often don’t require the full customer facing capacity and reliability.</p> <h4 id="offshoot--skunkworks--">Offshoot / Skunkworks <i class="icon ion-md-checkmark-circle glyp-green"></i></h4> <p>If there is a project or department that is separated from the rest you may not want it to run on your normal hosting solution. For those a PAAS might make sense. And later be rolled into the corporate infrastructure if the project itself becomes integrated.</p> <h4 id="personal-project-">Personal project <i class="icon ion-md-checkmark-circle glyp-green"></i></h4> <p>A PAAS is ideal for hobby projects. Something that normally can be on one node, with few visitors, and low resource requirements. It may have a database but that is about it.</p> <p>And many PAAS providers such as Heroku offers free hosting for low usage applications.</p> <h4 id="lost-of-personal-projects--">Lost of personal projects <i class="icon ion-md-close-circle glyp-red"></i></h4> <p>However if you have a lot of personal projects, and over time you will, you will exceed some free band restrictions with some PAAS providers.</p> <p>For instance Heroku offers <a href="https://devcenter.heroku.com/articles/free-dyno-hours">1000 free hours</a>, across all your applications combined.</p> <p>And whilst most of your inactive apps will sleep often, with enough apps, especially if some are hit often by e.g. Google search bots, then you will quickly exceed that limit.</p> <p>Then you will have to like me move some of the apps to another solution.</p> <h4 id="search-ranking-with-free-pricing-">Search ranking with free pricing <i class="icon ion-md-close-circle glyp-red"></i></h4> <p>Note if you use a PAAS providers free offering which uses a <em>sleep</em> feature, then you will experience slow response times when a request awakes it.</p> <p>This affects search rankings, and user experience.</p> <h4 id="resource-demanding-application-">Resource demanding application <i class="icon ion-md-close-circle glyp-red"></i></h4> <p>If your application demands a lot of processing power, or memory then a PAAS might not be suitable.</p> <p>PAASes often provides a limited node for free. E.g. 512mb with Heroku. And then offer more powerful nodes at a much higher price point.</p> <h4 id="core-business-">Core business <i class="icon ion-md-close-circle glyp-red"></i></h4> <p>If the application is your core business, and you are making a profit, then perhaps you should scale it out and invest in a custom infrastructure that perfectly suits your bespoke needs.</p> <h2 id="what-about-kubernetes">What about Kubernetes</h2> <p><a href="https://kubernetes.io">Kubernetes</a>, is the current big pink elephant in the room. The 300 pound gorilla. Whichever analogy I can wrongly apply, Kubernetes is very popular at the moment and relevant to a PAAS consideration.</p> <p>I am a heavy Kubernetes user myself. I have written some <a href="http://flurdy.com/docs/kubernetes/kubernetes-101.html">introduction to Kubernetes</a> docs on <a href="https://flurdy.com">flurdy.com</a> that may be helpful.</p> <p>Kubernetes bridges some of the gap between a pure IAAS and a PAAS. But it still requires significant time investment in training, configuration and normal operations monitoring and maintenance. And Kubernetes is very expensive as you end up running quite a few high memory nodes, with extra load balancers etc.</p> <p>Kubernetes is vast and complicated, with a large surface area of where it could go wrong. So it is not suitable for all, in every situation, all the time.</p> <h2 id="so-to-paas-or-not-to-paas">So, to PAAS or not to PAAS?</h2> <p>It depends.</p> <p>For a new project I would suggest just to initially throw it up on a PAAS. And not the free options, a proper supported <em>business</em> node.</p> <p>Validate it, iterate, then if it makes sense due to popularity or company policy move it to a custom infrastructure. And for that I would suggest Kubernetes.</p> <p>For any prototyping, just use a PAAS.</p> <p>For a large project I would suggest Kubernetes if you do not need to go live immediately. Especially if it ends up as multiple applications.</p>flurdyWhen is a PAAS the right tool?