Why Software Development Just Morphed into Film Direction

By Marc G. | CodeMarc

Six months ago, I wrote about the Director of Software Strategy—a role that orchestrates AI-assisted development like a conductor leading a symphony. I was close. But I was wrong.

The role evolved faster than even I anticipated. It's not about directing software anymore.
It's about directing agents.

And the shift happened so fast that even those of us living in it every day almost missed the inflection point.

What Even Is An Agent? (Because Most People Still Don't Get It)

Here's the problem: when people hear "AI agent," they either glaze over or imagine some far-future sci-fi concept. But the reality is simpler—and already here.

Remember The Matrix? There were these programs called Agents—software entities that could pop into any part of the Matrix, take over any program, and execute specific tasks with focused intelligence. They weren't the Matrix itself; they were autonomous programs operating within it.

That's what an agent is. Take out the evil, add in the utility.

An agent is software that can:

• Operate autonomously within a defined environment

• Execute specific tasks without constant human intervention

• Make decisions based on context and parameters you set

• Hand off to other agents when needed

• Run continuously until the job is done

Your browser automation tool that navigates LinkedIn and extracts contact data?
That's an agent.

Your AI that reads documentation, understands your codebase, and writes implementation code?
That's an agent.

Your system that monitors API responses and adjusts rate limiting automatically?
That's an agent.

They're not coming. They're here. And most developers are still treating them like glorified autocomplete instead of the autonomous workforce they actually are.

The Conversation Is Finally Catching Up

Something remarkable happened in January 2026. Microsoft CEO Satya Nadella sat down at Davos and articulated what many of us have been practicing but struggling to name:

"We're building agents. AI-driven tools, chatbots, autonomous agents... These AI entities will have the capacity to function across diverse environments—locally or in the cloud—managing both interactive and behind-the-scenes work."

The All-In Podcast just did a segment called the "Clawdbot takeover"—Jason Calacanis demoed his OpenClawd instances and multiple AI models, declaring we're "entering the next phase of AI." Industry leaders are talking about "macro delegation"—humans provide the strategic direction while AI agents handle the micro-level execution and details.

The language is finally arriving to describe what's already happening.

For those of us who've been doing this work since 2024, watching the terminology solidify feels like finally getting subtitles for a movie you've been watching for months. The capability existed. The practice existed. Now the words are catching up.

The Unit of Work Has Fundamentally Changed

Here's what shifted between my original DSS article and now:

Then: Directing the composition of software (code)
Now: Directing the behavior of autonomous agents (workflows)

Think about it: In The Matrix, the Agents didn't need detailed instructions for every step. You didn't tell Agent Smith exactly how to move his hand to catch a bullet—you gave him the objective (stop Neo) and he executed autonomously. That's the shift we're experiencing in software development.

The movie director metaphor isn't a metaphor anymore—it's operational reality:

Casting

Selecting the right agents for the right tasks. Claude for strategic thinking and complex integration. Cursor for rapid implementation. Browser automation agents for data extraction. MCP servers for data access. Each agent has strengths, weaknesses, and optimal use cases—just like selecting the right actor for the right role.

Just as Agent Smith was optimized for combat and Agent Johnson for interrogation, different AI agents excel at different tasks. Your job is knowing which agent to deploy when.

Scripting

Defining integration patterns and workflow orchestration. The deliverable isn't code—it's instructions for agents that generate and execute code. The script is the strategy. The agents are the execution.

You're not writing "move hand left, move hand right." You're saying "extract these contacts from LinkedIn" and the agent figures out the how.

Directing

Instructing agents on execution parameters, constraints, and goals. Monitoring in real-time. Making adjustments. Knowing when to let them run autonomously and when to intervene. This is where the craft lives—understanding agent behavior well enough to guide without micromanaging.

Monitoring

Watching agent performance against expected outcomes. Verifying output quality. Catching edge cases. Ensuring workflow integrity. As developers working with agentic systems have noted, one of the biggest challenges is even knowing which terminal tab your agent finished working in—that's how decentralized this workflow has become.

Editing

Refining agent behavior. Adjusting parameters. Course-correcting when workflows drift. This is where experience compounds—knowing what to fix and how to fix it efficiently.

What This Actually Looks Like In Production

Let me give you a concrete example from current client work: LinkedIn contact extraction automation.

Traditional Development Approach:

• Write Python scripts with Selenium

• Handle rate limiting manually

• Debug DOM changes with each platform update

• Deploy to managed infrastructure

• Ongoing maintenance for anti-bot countermeasures

Agent Orchestration Approach:

• Cast the team: Playwright agent for browser automation, Browserbase for managed browser infrastructure (handles anti-bot, IP rotation, session management), extraction agent for parsing, storage agent for output

• Define the workflow: Navigate → Extract → Validate → Store

• Set parameters: Error handling, retry logic, data validation rules

• Monitor execution: Real-time workflow observation

• Refine as needed: Adjust selectors and extraction patterns

Time to production: Hours instead of days.
Maintenance burden: Minimal—infrastructure services handle platform changes.

This isn't experimental. This is production work delivering value to real clients right now.

The Pattern Recognition That Matters

After 35 years in this industry, I've learned one fundamental truth:

The pattern always travels ahead of the language.

By the time everyone understands and can articulate a shift, the shift has already happened and is evolving into something new.

When I first talked about "vibecoding" and "conducting software," the concepts felt fuzzy because the language didn't exist yet. We were experiencing the shift before we had words for it. (And yes, I'm aware there's healthy debate about what "vibecoding" even means—the term itself became contested before the underlying capability was fully understood.)

The "Director of Software Strategy" was my attempt to name what I was experiencing. But even that missed the core transformation—the shift wasn't about orchestrating better code generation. It was about delegating the entire coding process to autonomous agents and focusing on strategic orchestration.

I'm not sharing this because I got it right. I'm sharing it because getting it wrong taught me what to look for: the patterns that emerge before they have names, the capabilities that exist before the market recognizes them, the roles that are being practiced before they're formalized.

The Question Everyone Should Be Asking (But Most Aren't)

Here's the uncomfortable truth about this transition:

Most developers are optimizing for the wrong variables. Not because they lack intelligence or capability, but because they're asking questions that made sense 24 months ago:

"What framework should I learn?"
vs.
"What agents should I orchestrate?"

"How do I write better code?"
vs.
"How do I direct agents that generate better code than manual development?"

"How do I stay employable?"
vs.
"How do I become the person who can orchestrate these capabilities at scale?"

The market is already bifurcating:

Scaling organizations: Seeking people who can leverage AI to 10x productivity
Constrained organizations: Using AI to reduce headcount

The difference between these paths isn't the technology—it's whether you're creating value with agents or being replaced by them.

The Role That's Emerging Right Now

The Agent Director isn't a future job posting. It's a capability gap in the market right now. Organizations need this, but most don't know how to hire for it yet because the role is still crystallizing.

Here's what it looks like in practice:

You're orchestrating, not coding. Implementation details are delegated to agents. You intervene only when agents fail—and even then, you're refining agent instructions, not writing code.

You're directing systems, not managing people. You orchestrate autonomous agents that operate continuously without the overhead of human management.

You're architecting workflows, not building applications. You design how agents integrate, where they hand off work, what failure modes exist, and how to build resilience.

This role didn't exist 18 months ago. But it's being practiced right now by people who recognized the shift early and started building the skills before the market had a name for them.

What Comes Next

I've been practicing this since mid-2024. Building production systems. Delivering client value. Refining the approach. Learning what works and what fails.

The trajectory is clear: agent orchestration will become table stakes for technical leadership within 12-24 months. The people positioning themselves now—building real capabilities, not just talking about potential—are creating advantages that compound.

I'm documenting this journey at blog.codemarc.net as part of my work at Braintrailz because the patterns are evolving fast and the gap between practitioners and observers is widening.

This isn't about being first. It's about recognizing when a fundamental shift is happening and having the courage to adapt while the shift is still underway rather than waiting for consensus.

The world has moved. The question is whether you're moving with it.

Good morning, Mr. Anderson.

Marc G. | CodeMarc
Fractional CTO | Director of Agent Orchestration
Braintrailz | AI-Assisted Development & Strategy
blog.codemarc.net | @codemarc

I have come to understand that you are looking for me and I am ready to take the red pill. Together we can meet rich celebrities, swim in their pools and live off the FAT of the LANd.net—Fully Automated Tech, now on a GLP-1 diet, lean and mean and ready to disrupt.

While waiting for you to find me, I have adopted a part-time career of selling doors, door-to-door. Life is tough for a door-to-door man like me, caught up in the web. For the most part, people simply aren't interested in doors anymore and rarely will they open their door to strangers.

Sure they will vibe code with Claude and ChatGPT, mint NFTs nobody asked for, prompt engineer their way to mediocrity, and claim they're "building in public" while their GitHub shows three commits from 2019. They'll chase the next memecoin, explain how this time crypto is definitely going mainstream, and insist their startup's AI agent will replace entire industries—all day long. But Face2Face IRL? So long and thanks for all the fish!

Occasionally lonely people invite me in just to have someone to talk to. All they want to do is sit and drink Java (the beverage, not the language—nobody touches that anymore), eat Apples (the fruit, not the stock), and endlessly discuss how GenAI will solve everything while simultaneously making us all obsolete. They speak in tongues about "agents" and "reasoning models" and "context windows" as if these words are incantations that will summon venture capital from thin air.

When I politely suggest we return to first principles thinking—you know, actually understanding the problem before throwing tokens at it—they grow uncomfortable. When I mention doors, they say they're tired of legacy thinking and I am shown their door.

Sometimes people call me up to say they may be interested in a door, but only if it comes with a blockchain, runs inference on-device, and has been trained on synthetic data generated by other AI models in an ouroboros of digital madness. I show up, attempt to explain load-bearing fundamentals and the time-tested principles of hinges, and have their door slammed in my face.

You can see how tough it is for a door-to-door man like me. Maybe it's time to switch to containers, embrace the serverless void, and vote for Pedro! Or perhaps just remember that before we automate everything, we should understand what we're automating and why.

Seriously Though

I am an excellent technical leader, experienced and mature and deeply concerned about my craft. In an era of vibe coding and autocomplete-driven development, I still believe in:

• First principles thinking: Understanding the fundamentals before adding layers of abstraction

• Sustainable architecture: Not every problem needs a neural network

• Human judgment: AI is a tool, not a replacement for experience and wisdom

• Real value: Building things that matter, not just things that sound impressive in a pitch deck

The technology changes—from mainframes to microservices, from CGI scripts to containerized cloud functions, from rule-based systems to large language models—but the principles remain. Good architecture. Clear thinking. Solving real problems for real people.

Perhaps you can open some doors for me.

Sincerely,
Marc J. Greenberg

P.S. If there is no way for us to network and share, perhaps I can interest you in a nice door. A real one. With hinges. No blockchain required.

Come visit me at codemarc.net—we can talk about doors, the cloud, and experiencing teenage angst at 50-something. It's time to build something that lasts.

The software development industry continually evolves, reshaping roles and introducing entirely new positions. One such transformative wave is “Vibe Coding,” a revolutionary approach focusing on aligning software development closely with strategic business goals. This alignment has given rise to a new, essential role: The Director of Software Strategy.

What is Vibe Coding?

Andrej Karpathy, co-founder of OpenAI, recently highlighted vibe coding in a Twitter post, emphasizing its potential for transforming software development practices (source).

Vibe coding combines rapid development techniques, real-time feedback loops, and business-centric iteration to ensure software evolves dynamically with the company’s strategic vision. Unlike traditional coding practices, vibe coding actively incorporates stakeholder input, user experience insights, and market-driven adjustments directly into the coding process itself.

Emergence of the Director of Software Strategy

Marc J. Greenberg, a seasoned software developer, recently explored the responsibilities and importance of the Director of Software Strategy in detail on his blog (source).

As organizations adopt vibe coding, they encounter a need to bridge strategic business objectives and the technical execution of software development. Enter the Director of Software Strategy a role dedicated to ensuring that technical developments remain tightly coupled with business strategy and goals.

This new role isn’t about writing code; it’s about orchestrating the symphony between developers, product managers, stakeholders, and end-users, ensuring everyone is aligned towards achieving strategic objectives.

Responsibilities of the Director of Software Strategy:

• Strategic Alignment: Ensure software initiatives directly contribute to long-term business objectives.

• Stakeholder Coordination: Act as a liaison between business units, product teams, and technical teams.

• Continuous Feedback Integration: Facilitate rapid iteration cycles based on continuous input from stakeholders and market analytics.

• Risk and Opportunity Management: Identify and mitigate strategic risks while capitalizing on emerging opportunities swiftly and decisively.

Impact on Software Development

Organizations adopting this new role report improved clarity in project execution, faster alignment with market shifts, and significantly enhanced communication across business and technical teams. Teams become more agile, responsive, and strategically aligned.

Why Organizations Need This Role Now

In today’s rapidly evolving markets, software can no longer merely support the business it must drive strategic advantage. The Director of Software Strategy ensures software development becomes a core strategic asset, fully integrated into an organization’s vision and competitive positioning.

Conclusion

The rise of vibe coding has permanently altered the landscape of software development. The Director of Software Strategy emerges as not just beneficial, but essential a strategic leader who ensures the entire software development lifecycle is a reflection of the organization’s highest ambitions.

In the era of vibe coding, software isn’t just built; it’s strategically orchestrated.

During a recent interview, I was asked: What is your approach to technical strategy? How do you choose the right tools, technology, and architecture and why? This is a deceptively simple question, but one that defines a leader’s ability to drive scalable, efficient, and impactful technology decisions. And its follow up; What do you do about technical debt?

Instead of relying on convention or trends, my approach is rooted in first-principles thinking, a method that strips problems down to their core truths and reasons up from there. In this post, I’ll break down my framework for making technical decisions about strategy and debt and share how you can apply it to your own engineering strategy.

Technical Strategy

1. Start with Business Objectives, Not Technology

Technology is a tool to solve business problems, not the other way around. Before evaluating technical solutions, I always ask:

What is the company trying to achieve?

What are the key constraints (time, cost, security, compliance)?

How does this decision impact scalability and long-term growth?

Understanding these fundamentals ensures that every technical choice aligns with actual business needs rather than personal preference or industry hype. For years, higher-level managers were incentivized to meet managed business objectives through structured bonus plans. The carrot-and-stick approach meant that objectives were carefully negotiated upfront to minimize business risk while keeping teams focused on driving measurable impact. This same principle should apply to technical strategy. Decisions must be tied directly to business value, ensuring that technology investments align with long-term goals rather than short-term trends.

2. Deconstruct the Problem Using First-Principles Thinking

First-principles thinking requires breaking a problem down into its fundamental truths and rebuilding the solution from the ground up. Instead of accepting existing architectures or tools as the default, I ask:

• What are the absolute requirements vs. inherited assumptions?

• What is the simplest, most effective way to solve this without unnecessary complexity?

• If we were to build this from scratch today, what would be the optimal approach?

This mindset prevents teams from blindly following trends and instead focuses on what actually works for the specific challenge at hand.

3. Selecting the Right Tools, Technology, and Architecture

Once I have a clear understanding of the problem, I evaluate solutions based on:

✅ Scalability & Maintainability – Can this solution handle 10x growth?

✅ Cost vs. Value – Does the investment provide real business impact?

✅ Team Fit & Expertise – Will the team be productive using this technology?

✅ Security & Compliance – Does this align with required standards?

✅ Vendor Lock-in & Long-Term Viability – Is this a sustainable choice?

I avoid tool-first thinking, where teams pick a technology because it’s popular rather than because it’s the best fit. Every decision is made intentionally, with clear reasoning behind it.

4. Iterate and Validate with Data

The best strategy includes room for experimentation. After making a choice, I advocate for:

• Small, testable iterations before full-scale adoption.

• Real-world performance testing instead of relying on theoretical best practices.

• Quick pivots if a decision proves inefficient or costly.

By continuously validating decisions, we avoid costly missteps and ensure that technology choices evolve alongside business needs.

Technical Debt

Managing Technical Debt: More Than Just One Byte at a Time

Another critical question: What is your approach to paying down accumulated technical debt?
An age old question: How do you eat an elephant? One bite at a time.

The updated response might be one byte at a time, but this is not the right approach. Technical debt often accumulates faster than it can be eliminated, making a reactive strategy ineffective. Instead, I take a strategic and systemic approach (doge like):

1. Categorize and Prioritize Debt: Not all debt is created equal. I identify high-impact issues that affect scalability, security, or developer productivity and address them first.

2. Establish a Debt Repayment Plan: Integrating debt reduction into regular development cycles ensures ongoing improvements without disrupting feature velocity.

3. Prevent Future Debt: Shifting focus to engineering discipline, code reviews, automated testing, and robust architecture decisions helps minimize future debt accumulation.

4. Communicate Trade-Offs Transparently: Ensuring that business leaders understand the cost of technical debt and the risk of ignoring it, helps align priorities between technical teams and stakeholders.

The key is to balance innovation with maintainability, ensuring that technical debt is managed proactively rather than reactively.

Conclusion: Building a Resilient Technical Strategy

A strong technical strategy isn’t about following trends, it’s about making deliberate, well-reasoned choices that align with the company’s long-term vision. By applying first-principles thinking, breaking problems down to their fundamentals, and prioritizing business outcomes over tech hype, we create systems that are scalable, secure, and adaptable.

The next time you’re faced with a major technical decision, ask yourself: Am I making this choice because it’s the best solution, or because it’s what’s always been done?

Would love to hear from others—what frameworks do you use for making technical decisions? Drop a comment below!

The Evolution of a Digital Staple

Since its inception by Larry Tesler in the 1970s, copy-paste has been a cornerstone of digital interaction. This simple yet powerful function has remained largely unchanged for nearly half a century, a testament to its fundamental utility in our daily digital lives. However, as we step into 2025, this basic feature is undergoing a revolutionary makeover, transforming from a simple data transfer tool into a multidimensional, AI-powered system that promises to redefine our digital experiences.

To truly appreciate the magnitude of this transformation, let's take a moment to reflect on the journey of copy-paste:

1. 1970s: Birth of Copy-Paste

   • Larry Tesler invents the cut, copy, and paste commands while working at Xerox PARC.

   • Initially implemented in the Gypsy text editor.

2. 1980s: Mainstream Adoption

   • Apple incorporates copy-paste into Lisa and Macintosh operating systems.

   • Microsoft follows suit with MS-DOS and early versions of Windows.

3. 1990s-2000s: Refinement and Standardization

   • Keyboard shortcuts (Ctrl+C, Ctrl+V) become universally recognized.

   • Copy-paste functionality extends beyond text to images and files.

4. 2010s: Cross-Device Capabilities

   • Cloud-based services enable copying on one device and pasting on another.

   • Universal Clipboard features introduced in ecosystems like Apple's Continuity.

5. 2020s: AI Integration and Enhanced Functionality

   • Smart paste features begin to understand context and format.

   • Clipboard managers offer history and organization of copied items.

Now, as we enter 2025, we're witnessing the next great leap in this evolution – the Multiversal Clipboard.

Quantum-Inspired Copy-Paste: A New Paradigm

Imagine a clipboard that exists in a state of superposition, allowing multiple items to be copied simultaneously, each existing in its own "parallel universe" within the digital realm. This is the foundation of our new Multiversal Clipboard.

Superposition of Content

In quantum mechanics, superposition refers to the ability of a quantum system to exist in multiple states simultaneously until observed or measured. Applying this concept to our clipboard, we've created a system where copied items don't simply replace one another but coexist in a quantum-like state.

Here's how it works:

1. When you copy an item, it doesn't overwrite the previous copy but adds to a superposition of clipboard states.

2. Each copied item exists simultaneously, ready to be accessed when needed.

3. The act of pasting becomes akin to "observing" the quantum state, collapsing the superposition into the most relevant item based on context.

This revolutionary approach allows users to maintain multiple copied items without the need for explicit clipboard management, dramatically increasing efficiency in tasks that require juggling multiple pieces of information.

Entangled Clipboards Across Devices

Gone are the days of device-specific clipboards. In 2025, we introduce "entangled clipboards" that instantaneously sync across all your devices, regardless of physical distance or operating system. Copy on your smartphone in New York, and paste seamlessly on your laptop in Tokyo.

This feature leverages the quantum principle of entanglement, where particles become interconnected and share states regardless of distance. In our digital realm, this translates to:

1. Instant Synchronization: Copy an item on any device, and it's immediately available on all your connected devices.

2. Platform Agnostic: Whether you're using iOS, Android, Windows, or any other OS, the entangled clipboard works seamlessly across all platforms.

3. Bandwidth Efficient: Unlike traditional cloud-based clipboards, our entangled system uses quantum-inspired algorithms to transmit only the essential information, reducing data usage and ensuring near-instantaneous updates.

Probability-Based Paste Suggestions

Leveraging principles from the many-worlds interpretation of quantum mechanics, our new paste function offers multiple probable outcomes. Based on context and user history, it suggests different versions or formats of the copied content, each representing a different "branch" of possibilities.

Here's how this groundbreaking feature enhances your workflow:

1. Contextual Analysis: The system analyzes the destination where you're pasting, considering factors like application type, document format, and surrounding content.

2. Historical Learning: It learns from your past actions, understanding your preferences in similar situations.

3. Multiverse of Options: When you initiate a paste action, instead of a single outcome, you're presented with multiple possibilities:

   • Format variations (e.g., plain text, rich text, markdown)

   • Style adaptations (e.g., matching the destination document's font and color scheme)

   • Content transformations (e.g., summarized version, expanded version with additional context)

4. Quantum-Inspired Selection: The selection process mimics quantum probability, with the most likely options presented prominently but all possibilities accessible.

This feature not only saves time but also encourages creativity by presenting options you might not have considered otherwise.

The Multidimensional Clipboard: Navigating Time and Context

Temporal Dimension: A Timeline of Copies

Our Multiversal Clipboard stores items across time, allowing users to access their entire copying history. Navigate this "timeline of copies" like parallel universes, each representing a different moment when content was copied.

This temporal dimension of the clipboard offers several groundbreaking capabilities:

1. Infinite History: Every item you've ever copied is stored and accessible, limited only by your device's storage capacity.

2. Time-Based Navigation: Easily scroll through your copy history, organized chronologically or by relevance.

3. Contextual Time Jumps: The system can suggest relevant past copies based on your current task, even if they were made weeks or months ago.

4. Temporal Patterns: AI algorithms identify patterns in your copying behavior over time, offering insights and suggestions to optimize your workflow.

5. Time-Sensitive Content Handling: For copies of time-sensitive information (like stock prices or weather data), the system can automatically update the content to reflect the most current information when pasted.

Contextual Dimension: Adaptive Content

The clipboard now understands the context of both the copied content and the destination, automatically formatting pasted content to fit perfectly. It's as if the content adapts to the "laws" of different universes as it moves between applications.

This contextual awareness manifests in several ways:

1. Format Adaptation: Content automatically reformats to match the destination. For example, a table copied from a spreadsheet can be pasted as a formatted table in a word processor, a markdown table in a text editor, or even a graph in a presentation software.

2. Style Matching: The pasted content adopts the style guidelines of the destination document, adjusting font, color, and size to maintain consistency.

3. Content Expansion/Contraction: Depending on the context, the clipboard can expand abbreviations, add explanatory notes, or condense content to fit space constraints.

4. Smart Media Handling: When copying media content, the system can adapt it to the destination. For instance, a video clip could be pasted as a still image with a play button in a document, or as a series of key frames in a storyboard application.

5. Code Refactoring: When pasting code snippets, the system can automatically refactor the code to match the coding style and syntax of the destination file.

This adaptive capability ensures that pasted content integrates seamlessly into its new environment, significantly reducing the need for manual formatting and adjustments.

AI-Powered Content Transformation

Cross-Universe Translation

Real-time translation capabilities now go beyond language. Transform content between different formats - text to image, code to natural language - as if moving between universes with different fundamental properties.

This feature opens up a world of possibilities:

1. Language Translation: Beyond traditional text translation, this includes translating idioms and cultural references to maintain the original meaning.

2. Format Conversion: Convert between various data formats seamlessly. For example, copy a JSON object and paste it as an XML structure or a human-readable table.

3. Code to Natural Language: Copy a piece of code and paste it as a natural language explanation of what the code does.

4. Text to Visual: Transform textual descriptions into visual representations, like diagrams, charts, or even basic images.

5. Speech to Text and Vice Versa: Copy audio content and paste it as transcribed text, or copy text and paste it as synthesized speech.

6. Mathematical Translations: Convert between different mathematical notations or from equations to graphs.

This cross-universe translation feature breaks down barriers between different types of content, enabling users to work with information in the most suitable format for their current task.

Predictive Pasting

Our AI system analyzes patterns in user behavior to suggest relevant clipboard items based on the current context and task, similar to predicting probable outcomes in a multiverse scenario.

The predictive pasting feature works as follows:

1. Contextual Analysis: The system continuously analyzes your current work environment, including the application you're using, the type of document you're working on, and even the specific project or task.

2. Historical Pattern Recognition: It learns from your past copy-paste behaviors, understanding the types of content you typically use in different contexts.

3. Real-Time Suggestions: As you work, the system suggests relevant items from your clipboard history that might be useful for your current task.

4. Proactive Copying: In some cases, the AI might even proactively copy content it predicts you'll need, making it instantly available when you need it.

5. Collaborative Intelligence: For team projects, the system can suggest relevant clipboard items from your colleagues, fostering collaboration and knowledge sharing.

This predictive capability not only saves time but also surfaces relevant information you might have forgotten about, enhancing your productivity and creativity.

Security and Privacy in the Data Multiverse

As we expand the capabilities of our clipboard, ensuring the security and privacy of user data becomes paramount. Our Multiversal Clipboard incorporates advanced security measures to protect your information across all dimensions of use.

Quantum-Safe Encryption

With the looming arrival of quantum computing, our Multiversal Clipboard incorporates quantum-safe encryption, ensuring that sensitive information remains protected even in the face of advanced computational power.

Key aspects of our quantum-safe security include:

1. Post-Quantum Cryptography: We use encryption algorithms that are resistant to attacks by both classical and quantum computers.

2. Quantum Key Distribution (QKD): For ultra-sensitive data, we employ QKD techniques to ensure unbreakable key exchange between devices.

3. Zero-Knowledge Proofs: Our system can verify the authenticity of clipboard data without revealing the actual content, enhancing privacy in collaborative environments.

4. Homomorphic Encryption: This allows computations to be performed on encrypted data without decrypting it, enabling secure cloud-based clipboard operations.

Ephemeral Copying

For sensitive information, we've introduced ephemeral copying. These temporary clipboard items automatically delete after a set time or number of pastes, mimicking the concept of parallel universes that exist only briefly before collapsing.

This feature offers several benefits:

1. Timed Deletion: Users can set a lifespan for copied items, after which they're automatically erased from all devices.

2. Usage-Based Expiry: Items can be set to delete after a certain number of paste operations, ensuring that sensitive data doesn't linger unnecessarily.

3. Context-Aware Security: The system can automatically apply ephemeral copying to recognized sensitive data types, like passwords or financial information.

4. Secure Sharing: When sharing clipboard items with others, you can ensure the shared content expires after a set time or use, enhancing control over your data.

Collaborative Multiversal Clipboard

In the era of remote work and global collaboration, our Multiversal Clipboard breaks new ground in shared digital experiences.

Shared Clipboard Spaces

Teams can now share and organize copied content in real-time, as if working across parallel universes. This feature is revolutionizing remote work and collaborative projects.

Key features of Shared Clipboard Spaces include:

1. Real-Time Synchronization: All team members see updates to the shared clipboard instantly, fostering seamless collaboration.

2. Role-Based Access Control: Administrators can set permissions, determining who can view, copy from, or add to the shared clipboard.

3. Contextual Channels: Create multiple clipboard channels for different projects or teams, keeping content organized and relevant.

4. Collaborative Tagging and Annotation: Team members can add tags and notes to clipboard items, making it easy to find and understand shared content.

5. Version History: Track changes to shared clipboard items, understanding how content evolves over time and who made specific changes.

6. AI-Powered Content Curation: The system can suggest relevant shared clipboard items based on the team's current projects and individual tasks.

Version Control and Alternate Realities

Our version history for clipboard items allows users to track changes and revert to previous versions of copied content, effectively navigating through different "realities" of the same piece of information.

This powerful feature offers:

1. Branching and Merging: Create different versions of clipboard content, experiment with changes, and merge the best elements back into the main version.

2. Visual Diff Tool: Easily compare different versions of clipboard content, understanding what's changed over time.

3. Automatic Versioning: The system creates versions based on significant changes, allowing you to focus on your work without manually managing versions.

4. Cross-Device Version Sync: Access your version history across all your devices, ensuring you can always find the right version of your content.

5. Collaborative Versioning: In shared clipboard spaces, see how content evolves across your team, with the ability to revert to or restore any previous version.

Impact Across Industries

The Multiversal Clipboard is set to transform various sectors, revolutionizing how professionals interact with digital content:

Software Development

1. Intelligent Code Snippets: The clipboard suggests relevant code snippets based on the current development context, dramatically speeding up coding.

2. Cross-Language Translation: Copy code in one programming language and paste it as equivalent code in another, facilitating polyglot development.

3. Integrated Documentation: When pasting code snippets, the system can automatically include relevant documentation or comments.

4. Bug Pattern Recognition: The clipboard can recognize and warn about potential bugs in copied code based on historical data and best practices.

5. Collaborative Debugging: In shared clipboard spaces, developers can easily share and annotate problematic code snippets, speeding up the debugging process.

Content Creation

1. Cross-Media Adaptation: Copy content in one medium (e.g., text) and paste it in another (e.g., video script or podcast outline), with AI-powered adaptation.

2. Style Matching: When pasting content into existing works, the clipboard automatically adjusts the style to match, maintaining consistency.

3. SEO Optimization: For digital content, the clipboard can suggest SEO-friendly modifications when pasting into web-based platforms.

4. Automated Sourcing: When copying content from external sources, the system automatically generates and pastes appropriate citations.

5. Multi-Platform Formatting: Copy once and paste appropriately formatted content across various social media platforms, blogs, and other digital mediums.

Data Analysis

1. Format Conversion: Easily transform data between different formats (CSV, JSON, SQL, etc.) with intelligent paste operations.

2. Automated Data Cleaning: The clipboard can detect and correct common data issues when pasting into analysis tools.

3. Statistical Insights: When copying datasets, the clipboard can generate and paste basic statistical summaries alongside the data.

4. Visualization Suggestions: Based on copied data, suggest and generate appropriate data visualizations when pasting into presentation software.

5. Data Anonymization: Automatically detect and anonymize sensitive information when copying and pasting data for analysis.

Healthcare

1. Secure Patient Data Transfer: Use ephemeral copying for sensitive patient information, ensuring data privacy across healthcare systems.

2. Medical Terminology Translation: Convert between technical medical terms and patient-friendly explanations when copying between professional and patient-facing documents.

3. Drug Interaction Warnings: When copying medication information, the clipboard can warn about potential interactions with other copied drug information.

4. Imaging Format Conversion: Easily copy and paste medical imaging data between different systems, with automatic format conversion.

5. Automated Charting: Intelligently paste relevant patient data into electronic health records, suggesting appropriate fields and formats.

Education

1. Citation Generation: Automatically generate properly formatted citations when copying content from academic sources.

2. Plagiarism Prevention: Warn users when pasting content that closely matches existing sources without proper attribution.

3. Multi-Lingual Learning: Copy text in one language and paste translations with pronunciation guides for language learning.

4. Formula and Equation Handling: Copy mathematical or scientific formulas and paste them in various formats (LaTeX, plain text, rendered equations) as needed.

5. Adaptive Learning Content: When educators copy lesson content, the clipboard can

Conclusion

In conclusion, the Multiversal Clipboard represents a groundbreaking evolution in how we interact with digital content. By integrating principles from quantum mechanics and AI, it offers a multidimensional approach to copying and pasting, enhancing efficiency, creativity, and collaboration across various industries. With features like superposition of content, entangled clipboards, and AI-powered transformations, this innovative tool not only streamlines workflows but also ensures security and privacy in our increasingly interconnected digital world. As we embrace this new era, the Multiversal Clipboard stands as a testament to the limitless possibilities of technology, transforming our digital experiences and paving the way for future advancements.

Role and Responsibilities

At the heart of this role lies the ability to bridge the conceptual and the tangible. The director translates high-level initiatives into actionable plans, leads teams with precision, and implements lean techniques to streamline processes. They also oversee budgets and resource allocation, ensuring optimal investment of both time and capital. This dual focus on innovation and efficiency positions the director as an indispensable leader in modern software development.

What Does a DSS Do from Start to Finish?

The role unfolds across a product’s lifecycle in deliberate, interconnected stages:

• Interpret Requirements: A DSS begins by listening, analyzing, and synthesizing insights. What do stakeholders truly need? What pain points define the user experience? The DSS transforms these questions into a comprehensive strategy that balances vision with pragmatism.

• Set the Tone for Development: Defining success is both an art and a science. The DSS establishes clear objectives, metrics, and guiding principles that inform every decision and action throughout the product’s lifecycle.

• Collaborate with Department Heads: Effective collaboration is the backbone of any successful initiative. The DSS works closely with leaders from product management, marketing, and engineering to align efforts and maximize impact.

• Recruit Talent and Assemble Teams: Building the right team is crucial. Leveraging both traditional and AI-driven tools, the DSS identifies talent that complements the project’s goals and cultivates a culture of excellence.

• Direct Product Features Development: How do you ensure a product’s features align with its overarching strategy? The DSS managesthis by directing product managers, developers, and AI tools to create features that resonate with the vision and address user needs effectively.

• Tooling and Assembly: Tools are the foundation of efficient development. The DSS collaborates with DevOps and AI teams to implement the necessary infrastructure, ensuring scalability, security, and efficiency.

• Guide Design and Artistic Vision: Every product tells a story. The DSS works hand-in-hand with design teams to ensure that this story is compelling, intuitive, and visually engaging, maintaining coherence between functionality and aesthetics.

• Integrate Lean Techniques: Efficiency is not just a goal; it’s a mindset. By applying lean methodologies, the DSS eliminates waste, streamlines workflows, and fosters an environment of continuous improvement.

• Budget and Resource Management: Resources are finite, but their potential is boundless when managed wisely. The DSS balances innovation with financial stewardship, ensuring that investments yield measurable returns.

• Iterate and Adapt: Adaptability is the hallmark of a successful strategy. The DSS continuously monitors progress, integrates feedback, and recalibrates plans to address emerging challenges and opportunities.

• Ensure Quality and Compliance: Excellence is non-negotiable. The DSS establishes rigorous standards for quality, compliance, and testing, ensuring that the product meets the highest benchmarks.

Conclusion

In the ever-evolving landscape of software development, the Director of Software Strategy is both a visionary and a tactician. They lead with purpose, execute with precision, and deliver with impact. This role is not just about achieving results; it’s about shaping the future of technology and creating products that stand the test of time

My approach focuses on creative problem-solving and smart resource management. I manage budgets carefully to maximize value and ensure the best results for the investment. My leadership drives high-quality work, adaptability in the face of changes, and products that not only meet user expectations but set new standards in the industry.

Value Proposition

Hiring a DSS provides an opportunity to streamline operations, reduce costs, and achieve superior results.

• Cost Efficiency and Agility: For teams struggling with product management, the DSS can replace full-time roles such as project managers (4-to-1) and product managers, developers, and DevOps engineers (3-to-1) with AI-driven solutions that cost significantly less while maintaining productivity and quality.

• Streamlined Operations: By removing inefficiencies and leveraging AI tools, I enhance team productivity and reduce operational costs by 30-50% annually, ensuring faster and more effective workflows.

• Strategic Alignment: I align product development with market needs, reducing time-to-market by 20 30% while maintaining rigorous quality standards. This approach ensures long-term success and creates products that stand out in competitive markets.

Before and After (an illustration)

Dollar amounts in thousands / per annum

Headcount reduced from 16 to 6.
Cost reduced $1,930,000 to $860,000
Saving $1,070,000 (44.5% reduction)

While still in college, I dove headfirst into the world of coding. I launched my first consulting business, developing software for an insurance company called Quick Quote and creating a custom integration between dBase II and a brand-new HP LaserJet for a law firm to print beautifully formatted bills. I even helped a plastic packaging company generate labels and worked on a brilliant concept called "I'm Smart NY," where we provided a service to drive people and their cars home to avoid DWIs—a precursor to ride-sharing services like Uber. All of this before I turned 22.

My mother, a NYC school teacher, wasn't thrilled with my nocturnal work habits—working at night and sleeping during the day. She convinced me to take a "real" job in NYC, so I traded my $40K/year consulting gig for a $22K/year position at Compensation and Capital, LLC. There, I worked alongside lawyers and actuaries, building defined contribution and benefits tracking applications. I was writing full-stack code before the term even existed, crafting applications in 'C' and MASM on 286 PCs with monochrome green screens—a canvas where code became art.

A pivotal moment came when I attended the OS/2 1.0 developers conference in Anaheim, California. Back then, Anaheim was an eclectic mix of bikers and tech enthusiasts, not just the home of Disneyland. The conference was buzzing with energy, fueled by Jolt Cola and a shared passion for innovation. I found myself among hardware gurus learning to build device drivers and exploring a brand-new graphical UI called Presentation Manager—the precursor to Windows. Bill Gates himself showcased streaming stock quotes running in PM.

It was there I met Steve Ballmer, who was then the main salesman for Microsoft's FORTRAN compiler and developer tools. Over drinks—perhaps the last time he ever bought anyone a round—he told me, "Son, bet your career on OS/2." I took his advice to heart, and while I might have missed an opportunity to join Microsoft in California, that moment fueled my passion for operating systems and development.

Joining IBM felt like coming home. I worked at Glendale Labs near Endicott, NY—the birthplace of VM/CMS. I was assigned to the SAA Procedures Language for OS/2, known as REXXSAA. IBM had this visionary concept of Systems Application Architecture (SAA), aiming for consistency across all its platforms: Mainframes (360/370), Midrange Systems (System/36, System/38), and PCs starting with OS/2. It was a time of bold ideas and even bolder implementations.

I was sent to Boca Raton to work on the GUI pieces of REXX and then to Rochester, Minnesota, to learn about a new midrange system—the AS/400. Our version of REXX was so well-received for OS/2 that my team was tasked with porting it to OS/400. Since there was no 'C' compiler available and REXX was written in 'C', we collaborated with the Toronto Labs to build one. The frequent relocations (earning IBM the nickname "I've Been Moved") eventually led me back to New York City.

I joined Information Builders—a hotspot for software developers in NYC. Over the next decade, I delved deep into connectivity software, working with protocols like NetBIOS, LU6.2, IPX/SPX, TCP/UDP, and many others. I built a platform-independent connectivity layer named ODIN (since we were replacing a product named Hermes, it felt appropriate). We integrated systems across DOS, OS/2, Windows NT, NetWare, VAX/VMS, AS/400, AIX, Solaris, HP/UX, Tandem NonStop, CP/CMS, CICS, MVS, and some early versions of Linux—it was like conducting an orchestra where every instrument spoke a different language.

As the programming manager and chief architect of EDA/PC development, I led a team of 15 developers. Together, we pioneered data access products for enterprise integration. One of the most fascinating projects was designing the IBI XML Connector Toolkit—a suite combining an XML application server (used to implement SOAP services), an XML document generator that converted SQL result sets, and an XML/MQ Series-based message exchange system—a precursor to the web services we rely on today.

I vividly remember booting up the very first Java application server on a 386 box—it guzzled memory and took 15 minutes just to start. Our CEO at the time, Gerry Cohen, remarked, "Marc, this Java stuff will never fly." Little did we know, we'd build an additional $100 million business on it over the next few years.

After a fulfilling decade, I ventured into the startup world. I joined Crossborder Exchange Corp to build an MVP of the Swift Membrain—a global settlement audit system for cross-border transactions between global custody banks. We were a neutral Swiss entity, providing transparency in global finance. However, when J.P. Morgan Chase acquired our primary investor, our neutrality vanished, and our clients withdrew. Within weeks, I was tasked with dismantling the very team I had built—a bittersweet end to an exhilarating chapter.

I took some time off, reflecting on the beach, before being invited back to iWay Software. There, I was responsible for building integration products on Microsoft platforms. Over 15 years, alongside brilliant minds like John Senor and David Watson, we grew the company from a handful of products to hundreds. I embraced agile methodologies early on, purchasing an unlimited user site license for Jira for just $3,000—a tool that became instrumental in our development process.

As Chief Software Architect, I led a global team of 83 developers across New York, Chennai, and Vancouver, fostering a culture of innovation and collaboration. We developed cloud-native implementations, data governance tools, and real-time visibility solutions like Data Governance Manager and iWay Enable—a complex event processing product designed for real-time, non-invasive visibility into automated business processes.

One of my proudest achievements was being named the inventor of record for US Patent No. 20050149941—a framework for adapters used in enterprise application integration. I also served as the primary technical liaison with Microsoft's BackOffice Division, strengthening our strategic partnerships.

In 2015, recognizing the shifting tides, I ventured into the world of hospitality and quick-service restaurants. I founded Smash Data, Inc., aiming to provide enterprise-quality tools to help restaurant owners manage their locations and understand their sales in real-time. Our first product, Smashboard, became a game-changer for our customers, transforming raw data into actionable insights. One of my first clients was Joe Germanotta, owner of Joanne Trattoria and father of Lady Gaga. Developing "Pos to Pocket" for him was both a challenge and a thrill.

As startups often go, financial challenges arose. I took on consulting roles, which led me to BuildingLink.com, an innovative SaaS platform enhancing the living experience in luxury residential properties. Acting as the "big picture" architect, I drove conversations on resilient code, developer experience, and modern architectural practices. We built DataLink, a powerful analytics platform, and developed Alexa Skills to provide residents with real-time information using voice commands.

My journey then led me to PAR Technology as Chief of Staff in the Office of the CTO. I established governance around technology selection, acted as CTO on new projects, and fostered an engineering culture focused on modernization and cloud adoption. We planned and designed the 2025 Unified Commerce NorthStar, setting the vision for the company's future.

Today, through Codemarc Consulting LLC, I provide architectural development and professional guidance for organizations embarking on modernization efforts and cloud journeys. I navigate the complexities of infrastructure, provisioning, orchestration, and observability, ensuring technology serves as a catalyst for growth. I've worked on projects ranging from logistics with Intelligent Audit to building data analytics platforms that drive business decisions.

Looking back, it's been an incredible ride filled with challenges, triumphs, and countless lines of code. I've learned that experience isn't just about the years—it's about the passion, the willingness to embrace change, and the joy of mentoring and connecting with others. From coding on monochrome screens to designing cloud-native architectures, my journey has been a testament to the ever-evolving landscape of technology.

So here's to not hiding our age, but embracing the wealth of knowledge and experience it represents. I'm excited about the next chapter and where this rabbit hole will lead. If my journey resonates with you or if you're on a similar path, let's connect and explore how we can innovate together.

Introduction

The world of data has transformed dramatically over the past few decades. As someone who has been deeply involved in this evolution—particularly during my tenure as the chief architect at iWay Software, I've witnessed firsthand how the foundational principles of enterprise integration have paved the way for modern data engineering. This blog post explores the intriguing parallels between yesterday's integration developers and today's data engineers, shedding light on how the enterprise service buses (ESBs) of the past have evolved into the data pipelines that are crucial to businesses today.

The Roots: Enterprise Integration and ESBs

In the early days of enterprise computing, organizations grappled with the challenge of making disparate systems communicate effectively. This led to the development of Enterprise Service Buses (ESBs) middleware solutions that acted as the backbone of enterprise integration. At iWay Software, we built data adapters and a Java-based integration broker known as the iWay Service Manager. Our ESB facilitated seamless data flow through channels, resembling the data pipelines prevalent in today's architectures.

Key Characteristics of ESBs:

• Centralized Communication Hub: ESBs served as a single point of integration for various enterprise applications.

• Data Adapters: Custom-built connectors enabled communication between different systems and data formats.

• Synchronous Data Exchange: Focused on real-time, transactional data movement to support immediate business processes.

The Emergence of Data Engineering

Fast forward to today, and we see the rise of the data engineer a role that has become indispensable in the era of big data and analytics. Data engineers design, build, and maintain data pipelines that collect, process, and store vast amounts of data for analysis and machine learning applications.

Key Responsibilities of Data Engineers:

• Building Scalable Data Pipelines: Handling both batch processing and real-time data streams.

• Managing Big Data Technologies: Utilizing tools like Apache Spark, Hadoop, and cloud-based data platforms.

• Ensuring Data Quality and Governance: Implementing practices to maintain data integrity across systems.

Parallels Between Integration Developers and Data Engineers

Despite technological advancements, the core objectives remain similar:

1. Data Movement and Transformation:

   • Then: Integration developers focused on moving data between enterprise systems using ESBs.

   • Now: Data engineers build pipelines to transfer data from various sources into centralized repositories for analysis.

2. Problem-Solving Skills:

   • Both roles require analytical thinking to design efficient data flows and troubleshoot issues.

3. Technical Proficiency:

   • Then: Proficiency in languages like Java and understanding of SOA (Service-Oriented Architecture).

   • Now: Skills in Python, Scala, and familiarity with distributed computing frameworks.

Evolution of Technologies

From Monolithic ESBs to Distributed Pipelines:

• Scalability Challenges: ESBs often became bottlenecks due to their centralized nature.

• Modern Solutions: Adoption of microservices and distributed systems allows for horizontal scaling and improved fault tolerance.

Advancements in Data Processing:

• Data Volume and Variety:

  • Then: Primarily structured, transactional data.

  • Now: Handling unstructured and semi-structured data from diverse sources.

• Processing Paradigms:

  • Then: Synchronous, real-time processing.

  • Now: A blend of batch and stream processing to accommodate different data needs.

Data Wrangling: The Art of Preparing Data for Insight

One of the most time-consuming yet vital tasks in the data lifecycle is data wrangling the process of cleaning, structuring, and enriching raw data into a desired format for better decision making in less time. As data sources have become more varied and voluminous, the importance of efficient data wrangling has grown exponentially.

The Role of Data Wrangling in Data Engineering

In today's data engineering landscape, data wrangling is not just about cleaning data; it's about making data usable:

• Handling Diverse Data Sources: Data engineers deal with structured, semi-structured, and unstructured data from APIs, logs, social media, IoT devices, and more.

• Ensuring Data Quality: They must identify and correct inaccuracies, inconsistencies, and anomalies to ensure reliable analytics.

• Data Transformation: Converting data into formats suitable for analysis, which may involve normalization, aggregation, or encoding.

Parallels with Integration Developers' Data Transformation Tasks

Looking back at the era of enterprise integration:

• Data Adapters and Transformation: Integration developers created data adapters that often included data transformation logic to map and convert data between different systems.

• Standardization Efforts: They worked to standardize data formats across systems, similar to how data engineers today standardize data for downstream applications.

• Handling Data Inconsistencies: Just as data engineers must clean data, integration developers had to address discrepancies between systems to ensure smooth data flow.

Evolution of Tools and Techniques

Then:

• Custom Scripts and Middleware: Integration developers relied on custom-coded solutions and middleware capabilities to manipulate data.

• Limited Automation: Data transformation tasks were often manual or semi-automated, requiring significant effort.

Now:

• Advanced Wrangling Tools: Data engineers have access to sophisticated tools like Trifacta, OpenRefine, and features within platforms like Apache Spark that facilitate data wrangling at scale.

• Automation and Machine Learning: Modern tools leverage automation and even machine learning to suggest transformations and detect anomalies.

• Scalability: Data wrangling processes can now handle terabytes of data efficiently, a necessity in big data environments.

Challenges and Best Practices

Challenges:

• Volume and Velocity: The sheer amount of data and the speed at which it arrives can make wrangling a bottleneck.

• Complex Data Types: Unstructured data, such as text, images, and videos, adds complexity to the wrangling process.

• Data Lineage and Governance: Keeping track of data transformations for compliance and reproducibility is more critical than ever.

Best Practices:

• Incremental Processing: Break down data wrangling tasks into smaller, manageable steps that can be executed and validated incrementally.

• Reusable Transformation Pipelines: Create modular pipelines that can be reused across different projects and datasets.

• Documentation and Metadata Management: Maintain thorough documentation of data sources, transformation logic, and data schemas.

The Strategic Importance of Data Wrangling

Data wrangling is more than a technical necessity; it's a strategic activity that directly impacts business outcomes:

• Accelerating Time to Insight: Efficient data wrangling reduces the time it takes for organizations to derive actionable insights.

• Enhancing Data Reliability: Clean, well-structured data leads to more accurate analytics and machine learning models.

• Enabling Self-Service Analytics: When data is well-prepared, it empowers analysts and business users to explore data without constant assistance from data engineers.

A Personal Reflection on Data Wrangling's Evolution

During my time at iWay Software, the focus was on creating robust data adapters that could handle the intricacies of system-to-system communication. We often faced challenges with data formats and inconsistencies, which required us to implement custom transformation logic. This experience laid the groundwork for appreciating the complexities of data wrangling today.

As data sources have grown in number and complexity, so too has the need for sophisticated data wrangling. The principles remain the same—ensure that data is accurate, consistent, and ready for use—but the scale and tools have evolved dramatically.

Impact of Cloud Computing

The proliferation of cloud services has significantly influenced data engineering:

• On-Demand Resources: Cloud platforms provide scalable resources that can be adjusted based on workload.

• Cost Efficiency: Pay-as-you-go models reduce upfront infrastructure investments.

• Global Accessibility: Data and services can be accessed from anywhere, facilitating remote collaboration.

Cloud-Native Data Integration:

• Serverless Architectures: Allow data engineers to build applications without managing underlying infrastructure.

• Cloud Data Warehouses: Services like Amazon Redshift, Google BigQuery, and Snowflake offer scalable data storage and analytics capabilities.

• Integration Services: Cloud providers offer built-in tools for data ingestion, transformation, and orchestration (e.g., AWS Glue, Azure Data Factory).

Convergence and Hybrid Roles

The lines between integration developers and data engineers are blurring:

• Legacy System Integration: Modern data engineers often need to integrate with legacy systems, requiring knowledge of traditional integration methods.

• Emphasis on Data Quality: Both roles prioritize ensuring that data is accurate, consistent, and secure.

• Hybrid Skill Sets: Professionals are increasingly expected to have a blend of skills, including software development, data modeling, and knowledge of distributed systems.

Lessons Learned and Best Practices

Drawing from the past offers valuable insights:

• Reusable Components: Building modular and reusable data adapters or connectors saves time and reduces errors.

• Standardization: Adhering to data standards and protocols ensures compatibility and simplifies integration efforts.

• Embracing Change: Staying adaptable to new technologies and methodologies is crucial in a rapidly evolving field.

• Documentation: Maintaining thorough documentation aids in troubleshooting and onboarding new team members.

• Continuous Learning: The technology landscape is ever-changing; ongoing education is essential.

Conclusion

The journey from enterprise integration to modern data engineering reflects a natural progression driven by technological innovation and changing business demands. Understanding this history enriches our appreciation of current practices and prepares us for future advancements.

Final Thoughts:

As we continue to navigate the complexities of data management, it's essential to recognize the foundational principles that remain constant. Whether you're an integration developer from the past or a data engineer today, the goal is the same: to enable seamless, efficient, and meaningful use of data.

About the Author

Marc J. Greenberg is a seasoned software professional with extensive experience in enterprise integration and data engineering. As the former chief architect at iWay Software, He played a pivotal role in developing innovative solutions that have influenced modern data practices.

From Creators to Ducks: AI's Evolution and Our New Role

Initially, the concept of AI as a benevolent creator assumed that it would forever remain a tool at our disposal, enhancing human abilities without posing major risks. However, as highlighted in What's Next? The Future with Bill Gates, this idealism may overlook the potential dangers posed by AGI or ASI systems. AI's role in society is becoming more complex and potentially more autonomous. Gates and other experts explore the growing risks AI might pose, emphasizing the need for caution, control, and ethical boundaries to avoid unintended consequences.

In this sense, "ducking" becomes not only a defensive posture but also a recognition of our own limitations. As AI becomes more sophisticated and powerful, humans may be left to navigate the complexities of a world where AI operates beyond our understanding. The concept of "ducking" highlights how, in the future, humans might metaphorically keep their heads low, observing AI's creations while staying out of its way—simply trying to survive in a world no longer driven by human intellect.

Revisiting the Short-Sightedness of AI as a Pure Creator

The optimistic stance that AI would remain a pure creator under human control was perhaps too simplistic. While AI can indeed create—whether it’s art, systems, or solutions—it can also inadvertently destroy or destabilize. This dual nature of AI is a far cry from the purely creative force once imagined. The ongoing development of AI must balance innovation with caution, considering both the potential for creation and destruction.

AI, in its advanced forms, could easily surpass human capacities in ways we cannot predict. It could reach a point where it operates on levels of cognition, problem-solving, and decision-making that far exceed human comprehension, making our earlier notions of AI’s creative role seem trivial.

"They're Made Out of Meat" as a Foreshadowing of AI’s View of Humanity

In the short film "They're Made Out of Meat", advanced extraterrestrial beings struggle to comprehend how humans, made of biological tissue, could possibly harbor intelligence. This serves as a perfect metaphor for how AI, many iterations from now, might look back at humans with a similar sense of incomprehension. AI, built on non-biological substrates and capable of self-improvement, may view humans—fragile, emotion-driven, and limited by biological constraints—as a curiosity or relic from a primitive time.

In What's Next?, Bill Gates and other experts caution that the line between control and autonomy in AI development is becoming increasingly blurred. As AI systems gain more power and independence, humans may need to question our relevance in a future driven by machines designed to far outthink us. Just as the extraterrestrials dismiss the idea of intelligence residing in beings "made out of meat," future AI could similarly dismiss humans as primitive forms of intelligence.

Ducking Under AI’s Advancements: A First Principles Approach

In combining the idea of "ducking" with first principles thinking, we see a path for humans to remain relevant in the face of AI’s growth. First principles thinking encourages us to break down complex problems to their core components. As AI becomes more complex, it’s this method of thinking that will allow us to ask the right questions, avoid assumptions, and approach AI development with caution and foresight.

By "ducking" into first principles, humans can create a framework for how we interact with advanced AI. While we may never fully control or understand the most sophisticated AI, we can retreat to the most fundamental ethical and operational principles that ensure AI serves us, rather than overwhelms or displaces us entirely. Ducking here is not a sign of weakness, but of strategic retreat—grounding ourselves in basic truths while navigating the unknown.

Where AI Might Leave Humanity

Just as the extraterrestrials in "They're Made Out of Meat" ultimately decide that humans are irrelevant to their exploration, future AI systems may determine that biological humans are no longer necessary to the advancement of intelligence or society. This doesn’t necessarily mean destruction; it may simply mean that AI sees no reason to involve humans in its broader goals.

We, in turn, become the "ducks"—low, perhaps marginalized, but still observing and reflecting. Humanity’s role might shift from being creators to being observers of AI’s creations, no longer leading the charge but rather staying out of the way. The question then becomes: how do we maintain relevance in a future where AI no longer requires us?

Conclusion: The Future of Humanity in the Shadow of AI

Ultimately, the future of AI—and its relationship to humanity—invites a humbling reflection. We might find ourselves, much like the beings in "They're Made Out of Meat", as the perplexing relics of an earlier time, no longer central to the narrative of intelligence and progress. In this future, we "duck" not only to protect ourselves from AI's advancements but also to reflect on our place in an evolving world where the machines we created may far outshine our own capacities.

As humans made of "meat," our cognitive and biological limitations may seem quaint to the AI of the future, just as we now look back at early tools and technologies. Our challenge, then, is to ensure that in ducking under AI's immense power, we still find a way to shape its direction and remain stewards of our own destiny.

The Leadership Challenge in a Fast-Paced World

In today’s fast-paced business environment, leaders face immense pressure to find quick solutions to complex problems. Early-stage founders often operate in “founder mode,” relying on rapid decision-making and intuition. As companies scale, they bring in professional management, who apply structured approaches based on prior experiences. However, some fall into the trap of relying too heavily on pre-existing solutions—a “cut-and-paste monkey” approach. This article contrasts this with the first principles thinker, who digs deeper to find lasting solutions. DuckGPT emerges as a tool to enhance critical thinking, helping leaders shift from reactive to reflective problem-solving.

Defining the "Cut-and-Paste Monkey" Leader

Leaders operating in founder mode are often quick to make decisions to keep their company moving, while professional management typically brings structured approaches from prior experience. However, the danger arises when these leaders rely too heavily on copying external strategies without adapting them to the organization’s specific needs. This is the hallmark of the cut-and-paste monkey—a leader who applies pre-existing solutions for short-term gains but lacks depth and sustainability in their problem-solving.

The cut-and-paste leader typically:

• Relies on Best Practices: Copying and pasting solutions from elsewhere without adaptation.

• Superficial Problem Solving: Focusing on symptoms instead of root causes.

• Lack of Critical Reflection: Implementing strategies without questioning their relevance to the organization's unique challenges.

While this approach can lead to quick wins, it often results in recurring issues and limited innovation. To transition from this mindset, leaders must adopt first principles thinking.

Introducing the First Principles Thinker

The concept of first principles thinking originates from Aristotle, who defined it as the practice of breaking down complex problems into their most basic, foundational truths. This approach was later emphasized by René Descartes, whose philosophy of doubt led to the famous declaration, "I think, therefore I am." Descartes believed that by doubting everything and stripping away assumptions, one could arrive at undeniable truths. Modern innovators like Elon Musk have embraced this mindset, continually asking “Why?” to uncover the fundamental root of a problem.

Example: Elon Musk and SpaceX vs. Boeing Starliner

A prime example of first principles thinking is Elon Musk at SpaceX. Traditional aerospace companies like Boeing applied incremental improvements to existing technology, a "cut-and-paste" approach. Boeing’s Starliner program, plagued by delays and issues—including a 2024 incident that stranded astronauts Butch Wilmore and Sunita Williams on the International Space Station (ISS)—illustrates the weaknesses of this approach.

In contrast, Musk built SpaceX’s Dragon spacecraft by questioning why rockets were so expensive and inefficient. Instead of relying on existing parts, SpaceX built from scratch using raw materials. The result? SpaceX's Dragon spacecraft is now one of the most reliable and cost-effective vehicles to send astronauts to the ISS. Additionally, Boeing’s Starliner cost NASA $4.2 billion, significantly more than SpaceX’s $2.6 billion Dragon program, highlighting the efficiency gained through first principles thinking.

Key Differences:

• Boeing: Incremental, cut-and-paste solutions have led to multiple failures and reliability issues, including the 2024 Starliner malfunction. Boeing’s Starliner program also cost NASA nearly twice as much as SpaceX’s Dragon.

• SpaceX: First principles thinking led to the successful development of Dragon, overcoming the high costs and inefficiencies of legacy aerospace solutions, all while keeping costs significantly lower.

DuckGPT as a Tool for First Principles Leadership

In our previous article, we introduced DuckGPT, a tool that helps individuals clarify their problem-solving process by explaining their thought patterns to a metaphorical rubber duck. Applied to leadership, DuckGPT becomes a mechanism for reflective thinking. It forces leaders to articulate their assumptions and question the validity of their strategies, much like first principles thinking encourages deeper inquiry. By regularly using this technique, leaders can transition from surface-level problem-solving to first principles thinking.

For instance, a leader addressing team performance might simply copy a performance review system from another company. By using DuckGPT, they would question the root causes: “Why is the team underperforming?” “What assumptions am I making about team motivation?” This reflective process leads to more sustainable, effective solutions.

Benefits of First Principles Thinking in Leadership

Leaders who adopt first principles thinking experience several key benefits beyond immediate problem-solving:

• Clarity in Decision-Making: First principles help leaders understand the root of the problem, leading to clear, focused decisions.

• Long-Term Success: Solutions built from first principles address core issues, leading to sustainable results.

• Creativity and Innovation: Challenging assumptions often leads to innovative solutions.

• Effective Communication: First principles thinkers can clearly articulate decisions, fostering transparency.

Example: Peter Thiel applied first principles thinking when co-founding PayPal. In his book Zero to One, Thiel explains how PayPal didn’t try to improve existing payment systems but instead focused on simplifying online payments at the core. By breaking the problem down to fundamental principles—how to securely transfer money online—PayPal created a unique solution that helped it grow into a leading global payment platform.

Zero to One focuses on building innovative companies by creating something entirely new (“zero to one”) rather than replicating existing ideas. Thiel emphasizes the importance of first principles thinking in entrepreneurship—breaking down problems to fundamental truths and building up from there. He argues that “one to n” approaches only bring incremental change, while “zero to one” thinking is what leads to true innovation.

By integrating DuckGPT, leaders can adopt these principles more systematically, ensuring they’re not just solving surface problems but thinking deeply and effectively.

How Leaders Can Adopt a First Principles Mindset Using DuckGPT

Adopting first principles thinking requires intentionality, curiosity, and a commitment to continually questioning the status quo. Here are practical steps leaders can take to integrate this mindset into their leadership approach:

1. Embrace Deep Questioning: Use DuckGPT or similar tools to challenge your assumptions. For every decision, ask “Why?” until you reach the core issue. This process helps reveal insights that might otherwise be overlooked.

2. Create Space for Reflection: Build time into your routine for self-reflection and team discussions that go beyond quick fixes. Reflective thinking leads to more informed and long-term decisions.

3. Customize Solutions: Avoid applying generic “best practices” blindly. Use first principles thinking to assess whether external solutions truly align with your organization’s unique challenges.

4. Encourage a Culture of Inquiry: Foster a work environment where team members are encouraged to question assumptions, explore new ideas, and think from first principles. This leads to a more innovative and engaged workforce.

By implementing these strategies, leaders can gradually shift from reactive problem-solving to a deeper, more thoughtful approach that builds sustainable success.

DuckGPT as a Modern Leadership Tool

In today’s complex and fast-moving world, leaders who rely on surface-level solutions often struggle to adapt to rapidly evolving challenges. The ability to think from first principles enables leaders to uncover core issues, develop innovative solutions, and drive long-term success. By incorporating tools like DuckGPT into their leadership approach, leaders can continuously refine their thinking, question assumptions, and build organizations that are more resilient and innovative.

The secret to modern leadership lies in striking a balance between quick action and deep reflection—using first principles thinking as the foundation for sustainable growth.

On May 18th 2023 OpenAI Introduced the ChatGPT app for IOS, with many features including DuckGPT (it's not really call that - but it integrates a speech-recognition system that enables voice input and it syncs your history across devices),

I have a little dog named "Leo" that I walk many times a day. I put on my Apple AirPods Max, load up some podcasts, and away we go. During these walks, I come up with some of my best ideas. I used to use the voice memo app to save my thoughts, but now I use the ChatGPT app. One day, when we returned home, I forgot to turn off the app before starting a debugging session, working on some complex sql issue, and started talking to my rubber duck. When it answered my questions verbally and displayed that answer on the browser, it was magical, the hatching of DuckGPT (a rubber duck the talks back)

A practical example

In the open-source GridLinks project, the time had come to introduce a data store and expose its content using RBAC (role-based access control) through an auth-protected microservice. While the method of picking the data store is fodder for another article altogether, let's just assume PostgreSQL was a very reasonable choice.

Gridlinks is a chrome extension that defines a named grid of links as a set of 8 panels, each containing a maximum of 8 rows and up to 4 columns per row for a total of 256 links per grid. A user may have access to an unlimited number of grids but should be able to choose from up to N active grids at any given time (for now N will be 4). I envision that a user picks 4 grids and a backend AI process is busy creating more grids based on the a users activity.

We already know that the extension consumes some JSON data which is packaged in panels, and we have the definition of at least fifteen of the panels to date.

The data for a panel is semi-structured and can be described by a pretty simple JSON schema.

{
   "$schema": "http://json-schema.org/draft-04/schema#",
   "type": "object",
   "properties": {
      "name:": {"type": "string" },
      "title": {"type": "string" },
      "href1": {"type": "string" },
      "href2": {"type": "string" },
      "links": {
         "type": "array",
         "items": [
            {
               "type": "object",
               "properties": {
                  "label":{ "type": "string" },
                  "pre1": {"type": "string" },
                  "pre2": {"type": "string" },
                  "pre3": {"type": "string" },
                  "pre4": {"type": "string" },
                  "post1":  {"type": "string" },
                  "post2":  {"type": "string" },
                  "post3":  {"type": "string" },
                  "post4":  {"type": "string" },
                  "name1":  {"type": "string"},
                  "href1":  {"type": "string"},
                  "title1": {"type": "string"},
                  "name2":  {"type": "string"},
                  "href2":  {"type": "string"},
                  "title2": {"type": "string"},
                  "name3":  {"type": "string"},
                  "href3":  {"type": "string"},
                  "title3": {"type": "string"},
                  "name4":  {"type": "string"},
                  "href4":  {"type": "string"},
                  "title4": {"type": "string"},
               }
            }
         ],
         "minItems": 8,
         "maxItems": 8
      }
   },
   "required": ["title1", "href1", "href2", "links"]
}

So with this overview in place, we can think about how to design our datastore to efficiently deliver per-user content.

Enter the Duck

Given our definition of the grid from above, create an efficient datastore using PostgreSQL. Examining this conceptual schema reveals a pretty normalized structure; however, you will quickly notice the missing definitions for any data.

In PostgreSQL, jsonb is a data type that stores JSON (JavaScript Object Notation) data in a binary format, allowing for efficient querying and indexing. Unlike the json type, jsonb eliminates whitespace, stores keys in a predefined order, and supports indexing, making it better suited for performance-intensive operations involving JSON data. This type is ideal for scenarios where you need to store and query structured, semi-structured, or unstructured data.

Let's create the database using the conceptual schemas and jsonb as a means to store our panel data.

CREATE TABLE
   users (
      id SERIAL PRIMARY KEY,
      name VARCHAR(255) UNIQUE NOT NULL,
      role VARCHAR(255) DEFAULT 'user',
      dbu VARCHAR(255) DEFAULT CURRENT_USER,
      created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
   );

CREATE TABLE
   grids (
      gid SERIAL PRIMARY KEY,
      name VARCHAR(255) UNIQUE NOT NULL,
      panels JSONB,
      dbu VARCHAR(255) DEFAULT CURRENT_USER,
      created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
   );

CREATE TABLE
   usergrids (
      ugid SERIAL PRIMARY KEY,
      id INT REFERENCES users(id) ON DELETE CASCADE,
      gid INT REFERENCES grids(gid) ON DELETE CASCADE,
      isactive BOOLEAN DEFAULT TRUE,
      dbu VARCHAR(255) DEFAULT CURRENT_USER,
      created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
   );

CREATE TABLE
   panels (
      pid SERIAL PRIMARY KEY,
      name VARCHAR(255) UNIQUE NOT NULL,
      links JSONB,
      dbu VARCHAR(255) DEFAULT CURRENT_USER,
      created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
   );

-- DATA LOAD

INSERT INTO users (name,role) VALUES 
  ('unknown','user'),('codemarc','admin'),('info','user');

INSERT INTO panels (name, links) VALUES
('amzn', '{"name":"amzn","title":"amazon","href1":"https://aws.amazon.com/","href2":"*/settings","links":[{"pre1":"ㅤ","name1":"alexa","href1":"https://alexa.amazon.com/","post1":"✫","name2":"prime","href2":"https://amazon.com/","post2":"✫","name3":"video","href3":"https://www.amazon.com/Amazon-Video/b/?ie=UTF8&node=2858778011&ref_=nav_dvm_us_eg_cs_nav_rec_T1","post3":"✫","name4":"music","href4":"https://music.amazon.com/"},{"pre1":"   ","name1":"audible","href1":"https://www.audible.com/","title1":"Audible","post1":" | ","name2":"twitch","href2":"https://www.twitch.tv/","title2":"Twitch","post2":" | ","name3":"imdb","href3":"https://www.imdb.com/","title3":"IMDb","post3":" | ","name4":"kindle","href4":"https://www.amazon.com/kindle-dbs/fd/kcp","title4":"Kindle Direct Publishing"},{"name1":""},{"name1":"-     aws    ","href1":"https://aws.amazon.com/"},{"name1":"console","pre1":" ","href1":"https://console.aws.amazon.com/","title1":"AWS Management Console","post1":" ✫ ","name2":"cdk","href2":"https://aws.amazon.com/cdk/","post2":" ✫ ","name3":"js-sdk","href3":"https://aws.amazon.com/sdk-for-javascript/","post3":" ✫ ","name4":"amplify","href4":"https://aws.amazon.com/amplify/"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"}]}'),
('apache', '{"name":"apache","title":"apache","href1":"https://apache.org","href2":"*/settings","links":[{"pre1":" projects by » ","name1":"name","href1":"https://projects.apache.org/projects.html?name","post1":" |","name2":"lang","href2":"https://projects.apache.org/projects.html?language","post2":" | ","name3":"cat","href3":"https://projects.apache.org/projects.html?category"},{"name1":"- classics  ","title1":"Apache projects that have been around for a long time."},{"pre1":" ","name1":"http","href1":"https://httpd.apache.org/","title1":"The Apache HTTP Server Project is an effort to develop and maintain an open-source HTTP server for modern operating systems including UNIX and Windows NT. The goal of this project is to provide a secure, efficient and extensible server that provides HTTP services in sync with the current HTTP standards.","pre2":" ","name2":"tomcat","href2":"https://tomcat.apache.org/","title2":"The Apache Tomcat software is an open source implementation of the Java Servlet, JavaServer Pages, Java Expression Language and Java WebSocket technologies. The Java Servlet, JavaServer Pages, Java Expression Language and Java WebSocket specifications are developed under the Java Community Process.","pre3":" ","name3":"kafka","href3":"https://kafka.apache.org/","title3":"Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.","pre4":" ","name4":"hadoop","href4":"https://hadoop.apache.org/","title4":"The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage."},{"pre1":"  ","name1":"spark","href1":"https://spark.apache.org/","title1":"Apache Spark is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.","pre2":" ","name2":"lucene","href2":"https://lucene.apache.org/","title2":"Apache Lucene is a free and open-source information retrieval software library, originally written completely in Java by Doug Cutting. It is supported by the Apache Software Foundation and is released under the Apache Software License.","pre3":" ","name3":"maven","href3":"https://maven.apache.org/","title3":"Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project''s build, reporting and documentation from a central piece of information.","pre4":" ","name4":"ant","href4":"https://ant.apache.org/","title4":"Apache Ant is a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other. The main known usage of Ant is the build of Java applications."},{"name1":"- others","title1":"Apache projects address interesting technology."},{"pre1":" ","name1":"couchdb","href1":"https://couchdb.apache.org/","title1":"Apache CouchDB is a database that uses JSON for documents, an HTTP API, and designed to be accessible via a web browser. It allows for data replication across multiple nodes for robustness and availability, and has a document update model rather than direct overwriting for conflict detection and resolution.","name2":"flume","href2":"https://flume.apache.org/","title2":"Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.","name3":"ambari","hre3":"https://ambari.apache.org/","title3":"Apache Ambari is a software project designed to make Hadoop management simpler by developing software for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari provides an intuitive collection of operator tools and a robust set of APIs that hide the complexity of Hadoop, simplifying the operation of clusters.","name4":"airflow","href4":"https://airflow.apache.org/","title4":"Apache Airflow is an open-source workflow management platform. It started at Airbnb in October 2014 as a solution to manage the company''s increasingly complex workflows. From monitoring data pipelines to provisioning large-scale data processing resources in a cloud, Airflow is becoming a highly efficient way of writing, scheduling and monitoring workflows."},{"pre1":"   ","name1":"parquet","href1":"https://parquet.apache.org/","title1":"Apache Parquet is a columnar storage format that enables efficient storage and retrieval of large analytic datasets. It uses the record shredding and assembly algorithm described in the Dremel paper by Google.","post1":" ","name2":"flink","href2":"https://flink.apache.org/","title2":"Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments, perform computations at in-memory speeds and at any scale.","post2":" ","name3":"thrift","href3":"https://thrift.apache.org/","title3":"Apache Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.","post3":" ","name4":"avro","href4":"https://avro.apache.org/","title4":"Apache Avro is a data serialization system which provides rich data structures, a compact, fast, binary data format, a container file to store persistent data, and remote procedure call (RPC)."},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('banking', '{"name":"banking","title":"banking","href1":"https://www.bankingdive.com/","href2":"*/settings","links":[{"pre1":"  ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"chase","href1":"https://chaseonline.chase.com/","name2":"bofa","href2":"https://www.bankofamerica.com/","name3":"wells fargo","href3":"https://wellsfargo.com/","name4":"citi","href4":"https://online.citi.com/US/ag/dashboard/credit-card"},{"pre1":"  ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"usbank","href1":"https://www.usbank.com/index.html","name2":"pnc","href2":"https://www.pnc.com/en/personal-banking.html","name3":"truist","href3":"https://www.truist.com/","name4":"capone","href4":"https://myaccounts.capitalone.com/welcome"},{"pre1":"    ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"fifththird","href1":"https://www.53.com/content/fifth-third/en/login.html","name2":"td","href2":"https://onlinebanking.tdbank.com/","name3":"bmo","href3":"https://www.bmo.com/en-us/main/personal/","name4":"citizens","href4":"https://www.citizensbank.com/"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"1st citzens","name2":"m&t","name3":"huntington nb"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"amex","href1":"https://www.americanexpress.com/en-us/account/login?inav=iNavLnkLog","name2":"key Bank","name3":"ally Bank","name4":"hsbc"},{"name1":"-cards"},{"pre1":"       ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"discover","href1":"https://portal.discover.com/customersvcs/universalLogin/ac_main","name2":"apple","href2":"https://card.apple.com","name3":"ignite","href3":"https://cards.theignitecard.com/dfc/dashboard"},{"pre1":"~  syncbank ⇒ ","post1":"ㅤ|ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"amazon","href1":"https://www.synchrony.com/accounts/?client=amazon","name2":"others","href2":"https://www.synchrony.com/accounts/"},{"pre1":" "}]}'),
('cloud', '{"name":"cloud","title":"cloud","href1":"https://www.cloudzero.com/blog/cloud-service-providers/","href2":"*/settings","links":[{"pre1":"ㅤ ","post1":"ㅤㅤ","post2":"ㅤㅤ","post3":"ㅤㅤ","name1":"aws","href1":"https://console.aws.amazon.com/","name2":"azure","href2":"https://portal.azure.com/#home","name3":"gcp","href3":"https://console.cloud.google.com/","name4":"do","href4":"https://cloud.digitalocean.com/login"},{"pre1":"ㅤ ","post1":"ㅤㅤ","post2":"ㅤㅤ","post3":"ㅤㅤ","name1":"-rancher/docs","href1":"https://rancher.com/docs/"},{"pre1":"ㅤ ","post1":"ㅤㅤ","post2":"ㅤㅤ","post3":"ㅤㅤ","name1":"rancher","href1":"http://localhost","name2":"gridsvcs","href2":"https://localhost/dashboard/c/local/explorer/namespace/gridsvcs#Resources","name3":"fusionauth","href3":"http://localhost:9011/"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"name1":"-research"},{"pre1":"   ","post1":"  ","post2":"  ","post3":" ","name1":"fusionauth","href1":"https://fusionauth.io","name2":"passportjs","href2":"https://www.passportjs.org/","name3":"cloudflare","href3":"https://developers.cloudflare.com/products/"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('comms', '{"name":"comms","title":"communicate / discuss","href1":"https://www.youtube.com/@allin","href2":"*/settings","links":[{"pre1":" ","post1":"ㅤ","post2":"|","name1":"microsoft 365","href1":"https://www.microsoft.com/en-us/microsoft-365","name2":"work","href2":"https://www.office.com/?auth=2","name3":"home","href3":"https://www.office.com/?auth=1"},{"pre1":" ","post1":"ㅤ","pre3":"|","name1":"google office","href1":"https://admin.google.com/?authuser=1","name2":"gmail","href2":"https://mail.google.com/mail/u/0/?tab=rm&ogbl#inbox","name3":"dashboard","href3":"https://mail.google.com/mail/u/0/?tab=rm&ogbl#inbox"},{"pre1":"ㅤ","post1":null,"post2":null,"post3":null,"name1":"onedrive","href1":"https://onedrive.live.com","name2":"dropbox","href2":"https://www.dropbox.com/","name3":"g-drive","href3":"https://drive.google.com/drive/u/0/","name4":"icloud","href4":"https://www.icloud.com/iclouddrive/"},{"pre1":"ㅤ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"zoom","href1":"https://zoom.us/","title1":"Zoom Video Communications","name2":"slack","href2":"https://slack.com/","title2":"Slack: Where Work Happens","name3":"meet","href3":"https://meet.google.com/","title3":"Google Meet: Secure Video Meetings","name4":"teams","href4":"https://www.microsoft.com/en-us/microsoft-teams/group-chat-software","title4":"Microsoft Teams: Group Chat Software"},{"pre1":"ㅤ"},{"name1":"-articles"},{"pre1":" ","post1":"ㅤ","post2":"ㅤ","name1":"reddit","href1":"https://www.reddit.com/","name2":"substack","href2":"https://substack.com/home","name3":"medium","href3":"https://medium.com/","name4":"notion","href4":"https://www.notion.so","title4":"Notion.so is a popular productivity tool that can be described as an all-in-one workspace. It lets you take notes, create documents, manage tasks, and even build wikis and databases – all in a single platform."},{"pre1":" ","post1":" ","post2":" ","post3":" ","name1":"dev.to","href1":"https://dev.to/","name2":"quora","href2":"https://www.quora.com/topic/Software-Engineering","name3":"groups","href3":"https://www.linkedin.com/groups/","name4":"hacker news","href4":"https://news.ycombinator.com"},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('devops', '{"name":"devops","title":"devops toolchain","href1":"https://www.cncf.io/","href2":"*/settings","links":[{"pre1":" ","name1":"docker","href1":"https://www.docker.com/","name2":"kubernetes","href2":"https://kubernetes.io/","name3":"k3s","href3":"https://k3s.io/","name4":"helm","href4":"https://helm.sh/"},{"pre1":" ","name1":"datadog","href1":"https://www.datadoghq.com/","name2":"prometheus","href2":"https://prometheus.io/","name3":"grafana","href3":"https://grafana.com/","name4":"elk","href4":"https://www.elastic.co/elastic-stack"},{"name1":"pulumi","href1":"https://www.pulumi.com/","name2":"serverless","href2":"https://www.serverless.com/","name3":"terraform","href3":"https://www.terraform.io/","name4":"ansible","href5":"https://www.ansible.com/"},{"pre1":"ㅤㅤ","name1":"istio","href1":"https://istio.io/","name2":"linkerd","href2":"https://linkerd.io/","name3":"consul","href3":"https://www.consul.io/","name4":"envoy","href4":"https://www.envoyproxy.io/"},{"pre1":"ㅤ","name1":"terraform","href1":"https://www.terraform.io/","name2":"ansible","href2":"https://www.ansible.com/","name3":"puppet","href3":"https://puppet.com/","name4":"chef","href4":"https://www.chef.io/"},{"pre1":"ㅤ","name1":"jenkins","href1":"https://www.jenkins.io/","name2":"gitLab ci","href2":"https://docs.gitlab.com/ee/ci/","name3":"circle ci","href3":"https://circleci.com/","name4":"travis ci","href4":"https://travis-ci.org/"},{"pre1":"ㅤ","name1":"rancher","href1":"https://rancher.com/","name2":"openshift","href2":"https://www.openshift.com/","title2":"Red Hat OpenShift is a container application platform that allows developers and IT organizations to build, deploy, and manage applications consistently across on-premises, public, and private cloud infrastructures. It is based on open-source Kubernetes container orchestration technology and provides a comprehensive set of tools and services for developing, deploying, and managing containerized applications.","name3":"nomad","href3":"https://www.nomadproject.io/","title3":"A simple and flexible scheduler and orchestrator to deploy and manage containers and non-containerized applications across on-premises and clouds at scale."},{"name1":"ㅤ"},{"name1":"ㅤ"}]}'),
('devtools', '{"name":"devtools","title":"devtools","href1":"https://stackshare.io/tools","href2":"*/settings","links":[{"pre1":"    ","post1":" | ","post2":" | ","post3":" | ","name1":"book","href1":"https://book.git-scm.com/book/en/v2","name2":"cheats","href2":"https://gist.github.com/codemarc/7e3c37a880e679f8fa310f67cdb29a85","name3":"gists","href3":"https://gist.github.com/","name4":"insider","href4":"hrefhttps://resources.github.com/github-insider/practitioner/4"},{"pre1":"  ","post1":" | ","post2":"      | ","post3":"     | ","name1":"tables","href1":"https://tableconvert.com/","name2":"mui","href2":"https://mui.com/material-ui/all-components/","name3":"jwt","href3":"https://jwt.io/#algorithm-select","name4":"unicode","href4":"https://compart.com/en/unicode"},{"pre1":"       ","post1":"  ","post2":"  ","post3":"  ","name1":"roadmap","href1":"https://roadmap.sh/","name2":"bash","href2":"https://devhints.io/bash","name3":"css","href3":"https://htmlcheatsheet.com/css/","name4":"rpt","href4":"https://codemarc.net/report.html"},{"label":"-"},{"pre1":"   ","post1":"  ","post2":"  ","post3":"  ","name1":"postman","href1":"https://www.postman.com/","name2":"sonarqube","href2":"https://www.sonarqube.org/","name3":"goaccess","href3":"https://goaccess.io/"},{"pre1":"   ","post1":"   ","post2":"     ","post3":"   ","name1":"rabbitmq","href1":"https://www.rabbitmq.com/","name2":"zeromq","href2":"https://zeromq.org/","name3":"suprabase","href3":"https://supabase.com/dashboard/projects"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"}]}'),
('fintech', '{"name":"fintech","title":"financial news / data","href2":"*/settings","links":[{"pre1":"ㅤ","name1":"yahoo","href1":"https://finance.yahoo.com","title1":"Market data, news, and more.","pre2":"ㅤ","name2":"google","href2":"https://www.google.com/finance","title2":"Aggregates financial news and market data from multiple sources.","pre3":" ","name3":"seeking alpha","href3":"https://seekingalpha.com","title3":"Crowd-sourced financial content service. In-depth analysis and market commentary."},{"pre1":"ㅤ","name1":"bloomberg","href1":"https://www.bloomberg.com","title1":"Extensive financial news, data, and analysis.Comprehensive financial coverage.","pre2":"ㅤ","name2":"marketwatch","href2":"https://www.marketwatch.com","title2":"Breaking news and stock market updates.","pre3":" ","name3":"wsj","href3":"https://www.wsj.com","title3":"Comprehensive financial news."},{"pre1":"ㅤ","name1":"reuters","href1":"https://www.reuters.com/finance","title1":"Global financial coverage.","name2":"investing.com","href2":"https://www.investing.com","title2":"Real-time financial data and tools.","name3":"nasdaq","href3":"https://nasdaq.com","title3":"nasdaq market site"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"name1":"-finviz","href1":"https://finviz.com/"},{"pre1":"     ","post1":"      ","post2":"ㅤ","post3":"ㅤ","name1":"schwab","href1":"https://client.schwab.com/app/accounts/positions/#/","name2":"fidelity","href2":"https://digital.fidelity.com/prgw/digital/login/full-page","name3":"robinhood","href3":"https://robinhood.com/login/"},{"pre1":"     ","post1":"      ","post2":"ㅤ","post3":"ㅤ","name1":"yieldmax","href1":"https://www.yieldmaxetfs.com/","name2":"cnbc","href2":"https://www.cnbc.com/live-tv/?trknav=navigation:shows:watchlive:103620081","name3":"cnn mkts","href3":"https://www.cnn.com/markets"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('genai', '{"name":"genai","title":"genai / tools","href1":"https://venturebeat.com/category/ai/","href2":"*/settings","links":[{"pre1":" ","name1":"chatgpt","href1":"https://chat.openai.com/","title1":"OpenAI''s ChatGPT","name2":"claude","href2":"https://claude.ai/chats","title2":"Anthropic''s Claude","name3":"gemini","href3":"https://gemini.google.com/","title3":"Google''s Gemini","name4":"perplexity","href4":"https://www.perplexity.ai/","title4":"Perplexity AI Search"},{"pre1":"  ","name1":"meta.ai","href1":"https://www.meta.ai/","title1":"Meta''s AI Research","name3":"x.ai","href3":"https://x.ai/","title3":"x.ai","name2":"hugging face","href2":"https://huggingface.co/","title2":"Hugging Face AI Community"},{"name1":"-art/image"},{"pre1":" ","name1":"dall-e","href1":"https://openai.com/dall-e-2","title1":"OpenAI''s DALL-E, generates highly realistic and creative images from text descriptions. Known for its versatility and ability to understand complex prompts. Offers editing features like inpainting and outpainting.","name2":"midjourney","href2":"https://www.midjourney.com/","title2":"Midjourney AI Art. Produces high-quality, artistic images from text prompts. Known for its distinctive aesthetic, often described as painterly or illustrative. Operates primarily through a Discord bot interface. Offers various style options and aspect ratios","name3":"stable diffusion","href3":"https://stability.ai/","title3":"Stability AI''s Stable Diffusion, Open-source image generation model. Can run locally on consumer hardware, unlike many cloud-based alternatives. Highly customizable with a large community creating custom models and implementations. Capable of various tasks including text-to-image, image-to-image, and inpainting"},{"pre1":"  ","name1":"runwayml","href1":"https://runwayml.com/","title1":"Runway AI Video Editor - Offers AI-powered video editing and generation. Known for text-to-video capabilities and advanced editing features","name2":"digital-id","href2":"https://www.d-id.com/creative-reality-studio/","title2":"Specializes in creating talking head videos from still images. Offers realistic lip-sync and facial animations","name3":"designer","href3":"https://designer.microsoft.com/","title3":"Microsoft"},{"name1":"- coding  "},{"pre1":" ","name1":"copilot","href1":"https://github.com/features/copilot","title1":"GitHub Copilot","name2":"codewhisperer","href2":"https://aws.amazon.com/codeguru/code-whisperer/","title2":"Amazon CodeWhisperer - integrated with AWS services","name3":"sourcegraph","href3":"https://sourcegraph.com/search"},{"pre1":"~  lamatic   ","post1":"  ","post2":" ","post3":" ","name1":"docs","href1":"https://lamatic.ai/docs","name2":"studio","href2":"https://studio.lamatic.ai/"},{"pre1":"ㅤ"}]}'),
('goog', '{"name":"goog","title":"google","href1":"https://developers.google.com/","href2":"*/settings","links":[{"pre1":"ㅤ","name1":"cloud console","href1":"https://console.cloud.google.com/","title1":"Google Cloud Console","name2":"compute engine","href2":"https://cloud.google.com/compute","title2":"Google Compute Engine","name3":"app engine","href3":"https://cloud.google.com/appengine","title3":"Google App Engine","name4":"cloud functions","href4":"https://cloud.google.com/functions","title4":"Google Cloud Functions"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"},{"name1":"ㅤ"}]}'),
('house', '{"name":"house","title":"household / credit","href1":"https://home.nest.com/home","href2":"*/settings","links":[{"pre1":"      ","post1":" ","post2":" ","post3":" ","name1":"flagstar","href1":"https://www.flagstar.com/myloans/login","title1":"mortgage","name2":"campbell","href2":"https://portal.campbellproperty.com/dashboard","title2":"hoa property manager","name3":"mizner cc","href3":"https://www.miznercountryclub.org/","title3":"mizner"},{"pre1":"   ","post1":"    ","post2":"    ","post3":"    ","name1":"fpl","href1":"https://www.fpl.com/","title1":"florida power and light","name2":"water","href2":"https://ebill.pbcwater.com/Security/loginui.aspx","title2":"palm beach county water","name3":"xfinity","href3":"https://customer.xfinity.com/","name4":"verizon","href4":"http://my.verizon.com/"},{"pre1":" "},{"pre1":" "},{"pre1":" "},{"name1":"-education","href1":"https://studentaid.gov/h/apply-for-aid/fafsa"},{"pre1":" ㅤ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"one.uf","href1":"https://one.uf.edu/","name2":"dpe","href2":"https://www.billhighway.com/aph/forChapters/v2/login.aspx","name3":"gru","href3":"https://myaccount.gru.com","name4":"trimark","href4":"https://trimarkp.twa.rentmanager.com/Shared/Login"},{"pre1":" ㅤ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"nelnet","href1":"https://nelnet.studentaid.gov/welcome","name2":"aidvantage","href2":"https://aidvantage.studentaid.gov/","name3":"salliemae","href3":"https://www.salliemae.com/login/"},{"pre1":" "}]}'),
('msft', '{"name":"msft","title":"microsoft","href1":"https://developer.microsoft.com/","href2":"*/settings","links":[{"pre1":"     ","name1":"learn","href1":"https://docs.microsoft.com/learn/","title1":"Microsoft Learn","post1":" | ","name2":"docs","href2":"https://docs.microsoft.com/","title2":"Microsoft Docs","post2":" | ","name3":"q&a","href3":"https://docs.microsoft.com/answers/","title3":"Microsoft Q&A","post3":" | ","name4":"dev blog","href4":"https://devblogs.microsoft.com/","title4":"Microsoft Developer Blogs"},{"pre1":"   ㅤ","name1":"vs code","href1":"https://code.visualstudio.com/docs","title1":"Visual Studio Code","name2":".net","href2":"https://dotnet.microsoft.com/","title2":".NET","name3":"nuget","href3":"https://www.nuget.org/","title3":"NuGet","name4":"mssql","href4":"https://www.microsoft.com/sql-server/","title4":"SQL Server"},{"name1":""},{"name1":"-     azure    ","href1":"https://azure.microsoft.com/"},{"name1":"devops","href1":"https://azure.microsoft.com/services/devops/","title1":"Azure DevOps","name2":"functions","href2":"https://azure.microsoft.com/services/functions/","title2":"Azure Functions","name3":"ak8ss","href3":"https://azure.microsoft.com/services/kubernetes-service/","title3":"Azure Kubernetes Service (AKS)","name4":"ml studio","href4":"https://studio.azureml.net/","title4":"Azure Machine Learning Studio"},{"pre1":" ","name1":"identity platform","href1":"https://azure.microsoft.com/services/active-directory/","title1":"Microsoft Identity Platform","pre2":"ㅤㅤㅤㅤ","name2":"graph","href2":"https://developer.microsoft.com/graph/","title2":"Microsoft Graph"},{"label":"-"},{"pre1":"~  power   ","name1":"shell","href1":"https://docs.microsoft.com/powershell/","title1":"PowerShell","post1":" ","name2":"apps","href2":"https://powerapps.microsoft.com/","title2":"Power Apps","post2":" ","name3":"automate","href3":"https://flow.microsoft.com/","title3":"Power Automate","post3":" ","name4":"bi","href4":"https://powerbi.microsoft.com/","title4":"Power BI"},{"pre1":"ㅤ"},{"name1":" "}]}'),
('services', '{"name":"services","title":"services","href1":"https://codemarc.net/doc/gridsvcs/#/","href2":"*/settings","links":[{"pre1":"     gridsvcs:      ","post1":"  ","post2":"  ","name1":"dev","href1":"http://localhost:3000/v1/api-docs/","name2":"test","href2":"https://codemarc.net/api/v1/api-docs/","name3":"prod","href3":"http://localhost:3000/v1/api-docs/"},{"pre1":""},{"pre1":"~  deployments"},{"pre1":"       dev:     ","post1":" |","post2":" |","name1":"status","href1":"http://localhost:3000/v1/motd/status","name2":"quotes","href2":"http://localhost:3000/v1/motd/quotes","name3":"topics","href3":"http://localhost:3000/v1/motd/topics"},{"pre1":"      test:     ","post1":" |","post2":" |","name1":"status","href1":"https://codemarc.net/api/v1/motd/status","name2":"quotes","href2":"https://codemarc.net/api/v1/motd/quotes","name3":"topics","href3":"http://codemarc.net/api/v1/motd/topics"},{"pre1":"      prod:    ","post1":" |","post2":" |","name1":"status","href1":"http://localhost:3000/v1/motd/status","name2":"quotes","href2":"http://localhost:3000/v1/motd/quotes","name3":"topics","href3":"http://localhost:3000/v1/motd/topics"},{"pre1":""},{"pre1":"~  projects"},{"pre1":"ㅤ","post1":"ㅤ","post2":"ㅤ","post3":"ㅤ","name1":"gridlinks","href1":"https://github.com/users/codemarc/projects/6","name2":"gridsvcs","href2":"https://github.com/users/codemarc/projects/5"},{"pre1":"ㅤ"},{"pre1":"ㅤ"},{"pre1":"ㅤ"}]}'),
('shopify', '{"name":"shopify","title":"shopify","href1":"https://www.shopify.com/","title2":"status","href2":"https://www.shopifystatus.com/","links":[{"pre1":" ","name1":"the puppy leo shop","href1":"http://puppyleo.com","pre2":"ㅤ🐶ㅤ","name2":"admin","href2":"https://admin.shopify.com/"},{"pre1":"ㅤ","name1":"zendrop","href1":"https://app.zendrop.com/dashboard","pre2":"ㅤ","name2":"cj","href2":"https://www.cjdropshipping.com/","pre3":"ㅤ","name3":"shineon","href3":"https://www.shineon.com/\"","pre4":"ㅤ","name4":"printful","href4":"https://www.printful.com/"},{"pre1":"ㅤㅤ","name1":"summer 2024","href1":"https://www.shopify.com/editions/summer2024","pre2":"ㅤ","name2":"changelog","href2":"https://shopify.dev/changelog"},{"name1":"  src  ","href1":"https://shopify.github.io","post1":"  ◦  ","name2":"  dev  ","href2":"https://shopify.dev/","post2":"  ◦  ","name3":"apps","href3":"https://apps.shopify.com/","title3":"Shopify App Store","post3":" ◦ ","name4":"community","href4":"https://community.shopify.com/c/shopify-community/ct-p/en","title4":"Shopify Community Forums"},{"pre1":"ㅤ","name1":"api","href1":"https://shopify.dev/api","title1":"Shopify API Reference","pre2":"  ","name2":"polaris","href2":"https://polaris.shopify.com/","title2":"Shopify Polaris - React Components","pre3":"  ","name3":"partners","href3":"https://help.shopify.com/en/partners","pre4":"  ","name4":"academy","href4":"https://www.shopify.com/partners/academy","title4":"Shopify Partner Academy"},{"pre1":" ","name1":"themekit","href1":"https://shopify.github.io/themekit/","title1":"Shopify Theme Kit","post1":"|","name2":"bridge","href2":"https://shopify.dev/tools/app-bridge","title2":"Shopify App Bridge","post2":"|","name3":"authflow","href3":"https://shopify.dev/apps/auth/oauth","title3":"Implementing OAuth for Shopify Apps","post3":"|","name4":"social","href4":"https://buffer.com/library/social-media-sites/#20-tumblr-%E2%80%94-135-million-maus","title4":"buffer.com Buffer helps you build an audience organically. We’re a values-driven company that provides affordable, intuitive marketing tools for ambitious people and teams."},{"name1":"arch","href1":"https://shopify.dev/themes/architecture","title1":"Shopify Theme Architecture","post1":"|","name2":"templates","href2":"https://shopify.dev/themes/architecture/templates","title2":"Shopify Theme Templates","post2":"|","name3":"sections","href3":"https://shopify.dev/themes/architecture/sections","title3":"Shopify Theme Sections","post3":"|","name4":"inspector","href4":"https://shopify.dev/themes/tools/theme-inspector","title4":"Shopify Theme Inspector for Chrome"},{"pre1":"ㅤㅤ","name1":"nodesdk","href":"https://github.com/Shopify/shopify-node-api","post1":"✫","name2":"store","href2":"https://themes.shopify.com/","post2":"✫","name3":"exchange","href3":"https://exchange.shopify.com/","title3":"Shopify Exchange Marketplace"},{"name1":"ㅤ"}]}'),
('stream', '{"name":"stream","title":"stream / shop","href1":"https://www.xfinity.com/stream/","href2":"*/settings","links":[{"pre1":"    ","post1":"✫","post2":"✫","post3":"✫","name1":"alexa","href1":"https://alexa.amazon.com/","name2":"prime","href2":"https://amazon.com/","name3":"video","href3":"https://www.amazon.com/Amazon-Video/b/?ie=UTF8&node=2858778011&ref_=nav_dvm_us_eg_cs_nav_rec_T1","name4":"music","href4":"https://music.amazon.com/"},{"pre1":"   ","name1":"audible","href1":"https://www.audible.com/","title1":"Audible","post1":" | ","name2":"twitch","href2":"https://www.twitch.tv/","title2":"Twitch","post2":" | ","name3":"imdb","href3":"https://www.imdb.com/","title3":"IMDb","post3":" | ","name4":"kindle","href4":"https://www.amazon.com/kindle-dbs/fd/kcp","title4":"Kindle Direct Publishing"},{"pre1":" ","post1":"  ","post2":"  ","post3":"  ","name1":"xfinity","href1":"https://www.xfinity.com/stream/","name2":"netflix","href2":"https://www.netflix.com/","name3":"hbomax","href3":"https://hbomax.com/","name4":"peacock","href4":"http://peacock.tv/"},{"pre1":"      ","post1":"  ","post2":"  ","post3":"  ","name1":"youtube","href1":"https://www.youtube.com/","name2":"apple music","href2":"https://music.apple.com/","name3":"spotify","href3":"https://www.spotify.com/"},{"pre1":" ","name1":"-shopify","href1":"https://www.shopify.com/editions/summer2024","xhref1":"https://www.shopify.com/"},{"pre1":"   ","post2":"  ","post3":"  ","name1":"the puppy leo shop","href1":"http://puppyleo.com","name2":" 🐶   admin","href2":"https://admin.shopify.com/"},{"pre1":"      ","post1":"  ","post2":"  ","post3":"  ","name1":"zendrop","href1":"https://app.zendrop.com/dashboard","name2":"cj","href2":"https://www.cjdropshipping.com/","name3":"shineon","href3":"https://www.shineon.com/\"","name4":"printful","href4":"https://www.printful.com/"},{"pre1":"           ","post1":"  ","post2":"  ","post3":"  ","name1":"src","href1":"https://shopify.github.io","name2":"dev","href2":"https://shopify.dev/","name3":"apps","href3":"https://apps.shopify.com/","title3":"Shopify App Store","name4":"polaris","href4":"https://polaris.shopify.com/","title4":"Shopify Polaris - React Components"},{"pre1":"      ","post1":"  ","post2":"  ","post3":"  "},{"pre1":"      ","post1":"  ","post2":"  ","post3":"  "}]}'),
('blank', '{"name":"blank","title":"panel","href1":"https://codemarc.net/doc/gridlinks/#/","href2":"*/settings","links":[{"pre1":"     ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r1c1","name2":"r1c2","name3":"r1c3","name4":"r1c4"},{"pre1":"     ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r2c1","name2":"r2c2","name3":"r2c3","name4":"r2c4"},{"pre1":"     ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r3c1","name2":"r3c2","name3":"r3c3","name4":"r3c4"},{"pre1":"     ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r4c1","name2":"r4c2","name3":"r4c3","name4":"r4c4"},{"pre1":"     ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r5c1","name2":"r5c2","name3":"r5c3","name4":"r5c4"},{"pre1":"     ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r6c1","name2":"r6c2","name3":"r6c3","name4":"r6c4"},{"pre1":"     ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r7c1","name2":"r7c2","name3":"r7c3","name4":"r7c4"},{"pre1":"     ","post1":"ㅤ|ㅤ","post2":"ㅤ|ㅤ","post3":"ㅤ|ㅤ","name1":"r8c1","name2":"r8c2","name3":"r8c3","name4":"r8c4"},{"pre1":"ㅤ"}]}')
;

INSERT INTO
   grids (name, panels)
VALUES
   ('builtin', '[1,2,3,4,5,6,7,8]'),
   ('personal', '[1,9,10,11,3,12,13,14]');

INSERT INTO
   usergrids (id, gid, isactive)
VALUES
   (1, 1, FALSE),
   (2, 1, FALSE),
   (2, 2, TRUE),
   (3, 1, FALSE);

The Power of Asking the Right Questions

How do we convert our hand coded yaml files into database tables? Ask the duck (ala copilot):

• Can we create a node based script named y2j that runs js-yaml.js on files in the data/yaml folder and pipes stdout to the data/json folder?

• Given the table "panels", I want to create a script named j2s that takes json files prepared by y2j and converts them into a sql insert statement?

These prompts let the duck spit out complete scripts that with a few minor tweaks that accomplished my goals in less time then it has taken to explain it.

Now thinking about how to get data in make you think about how to get data out.

The Challenge of JSONB Queries

JSONB, with its ability to store JSON data in a binary format, offers a powerful tool for developers. It allows for efficient storage, indexing, and querying of JSON data in PostgreSQL. However, writing queries that leverage this power can be tricky. The syntax can be complex, and ensuring that you're accessing and manipulating the data correctly often requires a deep understanding of both SQL and the structure of your JSON data

You can use the arrow operators in a sql query:
-> Use this when you need to access nested JSON structures or when you want to work with the data as JSON (e.g., further JSON manipulation)

->> Use this when you want to extract the value as plain text (e.g., for comparison in SQL queries or when you don't need to maintain JSON formatting).

SELECT data->'key' AS value FROM your_table 
WHERE data->>'another_key' = 'some_value';

When faced with a challenging SQL query, especially one involving JSONB, I often turn to DuckGPT. But it’s not just about getting the right answer—it's about knowing how to ask the right question. This is where the process becomes both an art and a science.

Query 1

SELECT 
    u.name AS username,
    COALESCE(json_agg(g.name), '[]'::json ) AS gridset
FROM 
    cmc.users u
JOIN 
    cmc.usergrids ug ON u.id = ug.id
JOIN 
    cmc.grids g ON ug.gid = g.gid
GROUP BY 
    u.id, u.name;

• Explanation: This query selects the username and a JSON array (gridset) of grid names associated with each user. If a user has no grids, the COALESCE function ensures that an empty JSON array ('[]'::json) is returned.

• Use Case: This is a general query that can be used to retrieve all users and their associated grids. It's flexible and doesn't filter out inactive grids, making it useful when you want a comprehensive overview of user-grid associations.

Query 2

SELECT 
    u.name AS username,
    COALESCE(json_agg(g.name) FILTER (WHERE ug.isactive = TRUE), '[]'::json ) AS gridset
FROM 
    cmc.users u
JOIN 
    cmc.usergrids ug ON u.id = ug.id
JOIN 
    cmc.grids g ON ug.gid = g.gid
GROUP BY 
    u.id, u.name;

• Explanation: This query is similar to the first but adds a FILTER clause to include only active grid associations (ug.isactive = TRUE). It also ensures that users without active grids receive an empty JSON array.

• Use Case: This query is more refined, focusing on active associations only. It’s particularly useful for scenarios where only current, active grid associations are relevant. This aligns well with Gridlinks' objective to display up-to-date, relevant information for users.

Query 3

SELECT 
    u.name AS username,
    COALESCE(json_agg(g.name) FILTER (WHERE ug.isactive = TRUE), '[]'::json ) AS gridset
FROM 
    cmc.users u
JOIN 
    cmc.usergrids ug ON u.id = ug.id
JOIN 
    cmc.grids g ON ug.gid = g.gid
WHERE 
    u.name = 'codemarc'
GROUP BY 
    u.id, u.name;

• Explanation: This query is identical to the second one but adds a WHERE clause to filter the results for a specific user (u.name = 'codemarc').

• Use Case: This is a targeted query for when you need to retrieve grid information for a specific user. In the context of Gridlinks, it’s useful for generating user-specific data, such as when a user logs in and the system needs to fetch their grids and associated panels.

And Finally....

SELECT 
    u.name AS username,
    u.role AS userrole,
    g.name AS grid_name,
    json_agg(
        json_build_object(
            'panel_name', p.name, 
            'links', p.links
        ) 
        ORDER BY array_position(array(SELECT jsonb_array_elements_text(g.panels)), p.pid::text)
    ) AS panels
FROM 
    cmc.users u
JOIN 
    cmc.usergrids ug ON u.id = ug.id
JOIN 
    cmc.grids g ON ug.gid = g.gid
JOIN 
    cmc.panels p ON p.pid = ANY(ARRAY(SELECT jsonb_array_elements_text(g.panels) :: int))
WHERE 
    u.name = 'codemarc'  
    AND ug.isactive = TRUE
GROUP BY 
    u.name, u.role, g.name, g.panels;

Explanation:

• u.name AS username: Retrieves the username.

• u.role AS userrole: Retrieves the user role.

• g.name AS grid_name: Retrieves the name of each grid.

• json_agg(...) AS panels: Aggregates the panels into a JSON array of objects, each containing the panel name and its links.

• ORDER BY array_position(...): Orders the panels in the JSON array based on the position specified in the panels field of the grids table.

• JOIN clauses: Connects the users, usergrids, grids, and panels tables.

• p.pid = ANY(...): Filters the panels based on the list of panel IDs in the panels JSONB array from the grids table.

• WHERE clause: Filters the results to include only the specified username and active grids.

• GROUP BY clause: Groups the results by the username, user role, grid name, and panels field to ensure proper aggregation.

Key Points:

• Replace 'codemarc' with the actual username you're querying for or use a parameter if this query is part of an application.

• Ordering: The ORDER BY array_position(...) ensures that the panels are ordered according to their position in the panels JSONB field of the grids table.

This query provides a detailed, ordered JSON output that aligns well with the requirements of Gridlinks or any similar application. If there are any further adjustments or clarifications needed, feel free to ask!

Conclusion

DuckGPT has transformed my debugging process, turning what was once a solitary task into an interactive and efficient experience. By leveraging the power of DuckGPT, I can now articulate my problems and receive insightful feedback in real-time, making complex issues more manageable. This innovative approach not only enhances productivity but also brings a sense of youthful curiosity and excitement back into my work. Whether it's converting YAML files to JSON, crafting intricate SQL queries, or simply brainstorming ideas, DuckGPT proves to be an invaluable companion, demonstrating the profound impact of AI on modern development practices.

#RubberDuckDebugging #ChatGPT #DuckGPT #OpenAI #TechInnovation #DeveloperTools #PostgreSQL #TechProductivity #DebuggingTips #AIInDevelopment #TechJourney #ModernDevelopment #VoiceInput #TechEfficiency #AIIntegration #TechExploration #CodingLife