I'm CA Lee, a solo developer focusing on [[Data Engineering.png|Data Engineering]].
I started by learning SQL on my own time at the company that I work for, then Python (pandas) for data analysis and eventually picked up other development skills along the way (see [[05-13_Evolution]])
# Tools
> [!NOTE] Tech should be employed to solve an actual problem.
> I've spent a lot of time debating what works and what doesn't, and also had to backtrack several times as I chose to work with tools that were way out of proportion with the scale of the problems that I worked with.
- IDE - Windsurf / Zed
I must have tried like a dozen editors by now and I really like Zed for its _speed_ and _simplicity_ - using it feels like Sublime Text with AI features (Zed works closely with Anthropic). However, for developers, the ecosystem is not mature enough yet. For example, I work a lot with SQL Server and dbt and VS Code has much more mature tooling than Zed has.
Windsurf, being a VS Code fork, is still my preferred daily driver for now. It has *super fast and accurate tab completion* which I found to be superior to Zed's at this point in time.
I haven't used Cursor enough to comment on it, even though it seems to be the de facto weapon of choice amongst Twitter devs. They recently got upgraded context on larger codebases, so its worth dropping by from time to time to see how it has improved or evolved. Also, they probably have the most mature tooling amongst the AI enabled IDEs, with `.devcontainer` support, amongst other extensions.
Cursor - $10bn
Windsurf - $3bn
Zed - ?
It feels like the Cursor has the first mover advantage, being the OG VS Code fork, but once Windsurf exits via OpenAI funding, I think Zed's hunger (from not having cashed out yet) and overall developer philosophy should win in the long run.
- Databases
- SQL Server (mainly for work)
- supabase
- duckdb (interested in MotherDuck data warehouse)
- Documentation
- Notion - Unparalleled for publishing WYSIWYG websites
- Obsidian - I use it more as a personal thinking tool, although I liked the CEO's philosophy enough to buy their Sync and Publish products.
Other tools that I work with:
- AppSheet
- Data modeling (dbt)
- Data notebooks (Jupyter, marimo)
- Data applications (Streamlit)
- Data Visualisation, Business Intelligence (Metabase)
- Deployment (Docker, Linux)
# Architecture
Many hosted services can be simplified with just renting a VPS with a very modest monthly cost.
```mermaid
graph LR
%% Define subgraphs for grouping
subgraph hc["`Hetzner Cloud
4 vCPU, 16GB RAM`"]
MB(Metabase)
ST(Streamlit)
NEXT(Next.js)
end
subgraph OnPrem[On-Premises]
SQL(SQL Server)
end
subgraph SaaS[Cloud Services]
SB(Supabase)
GCS(GCS)
APP(AppSheet)
GS(Google Sheets)
end
%% Define relationships
SQL --> MB
SQL --> ST
SQL --> APP
SQL --> GS
SB --> MB
APP --> GCS
%% Bidirectional (apps)
NEXT <--> SB
ST <--> SB
%% Style classes for Obsidian linking
class SQL,GS,MB,ST,SB,GCS,APP,NEXT internal-link;
%% Color scheme
classDef onPrem fill:#2c3e50,stroke:#34495e,color:#ecf0f1
classDef hetzner fill:#2980b9,stroke:#3498db,color:#ecf0f1
classDef cloudServices fill:#8e44ad,stroke:#9b59b6,color:#ecf0f1
%% Apply styles
class SQL onPrem;
class MB,ST,NEXT hetzner;
class GS,SB,GCS,APP cloudServices;
%% Style subgraphs
classDef subgraphStyle fill:none,stroke:#607d8b,stroke-width:2px;
class Hetzner,OnPrem,Cloud subgraphStyle;
```