Hey everyone,
Those of you following along for the last few weeks know about my plan to help my mom migrate her significant Microsoft Word document collection (which is mostly notes) to a more purpose-built app actually built for taking and organizing notes.
You also know that my original intent was to use Craft, but then I encountered an unexpected hiccup with a not-100%-ready important feature, and I started second-guessing my choice. As I explained in last week's message, I finally decided instead to use a specifically pre-configured Obsidian installation--not something I would recommend for everyone, but I'm confident in my ability to set things up so she can use it efficiently without getting lost in the powerful but complicated flexibility that Obsidian provides.
Time will tell if I'm right about this.
As it happens, my mom is on a trip this week, so she and I couldn't get further into the migration process together. But that doesn't mean I can't talk about the process! And, I have something really awesome to show you, too--at least, I think it's awesome. (SPOILER: local AI-assisted document analysis and organization!) It's pretty nerdy, but...well, you'll have to decide for yourself.
But first, how does one actually "migrate a bunch of documents into a note-taking app," as I've stated our goal to be?
Let's look at the current situation and identify the most important problems to solve:
- Thousands of individual Word documents currently reside in a complex folder structure with rampant topical overlap.
- Knowing where to put new documents is not always obvious.
- Knowing where to find old documents is even less obvious.
- Knowing what content is in each document requires individually opening and reading that file--cumbersome and time-consuming.
- There is no easy way to search all documents for a specific topic, person, or category of content with any reliability.
Those are the main problems, though we could probably come up with more. If you're someone who uses Word (or any similar tool) to collect information the way my mom has for the last decade or so, this might sound familiar. Or, maybe you're responsible for a bunch of business documents at your workplace, and they are similarly (dis)organized--the problem changes only superficially.
So what are we aiming for as a solution? Well, just look at the problem statements and identify the opposite situation:
- All documents are in a simple, reasonably "flat" or "shallow" folder structure that has little to no ambiguity or redundancy.
- New documents belong in an obvious place based on the intended content type and/or topic (e.g. "meeting note" or "journal entry").
- Old documents are found in equally obvious locations without requiring a manual hunt and endless scrolling through hundreds of folders.
- Each document's content can be inferred at a high level based on its location, and at a detailed level based on tags, backlinks, and summaries.
- All notes have at least a few links and tags to relevant topics, people, or organizations, allowing an efficient Map Of Content (MOC) organization.
It's a tall order, but 100% achievable with a good process.
Before we can make significant progress, we have a couple of different questions to answer. Fortunately, we've already answered one of them: what app or platform do we want to use? In our case, we're going ahead with Obsidian.
The other important question concerns which documents we actually want to migrate into the new system. A couple of weeks ago, I listed three options:
- Approach A - The Skeptic's Trial Run: Keep using Word, but duplicate a few notes in the new app to see how it works first.
- Approach B - The Clean Break: Use the new app for everything going forward, and keep Word around to access your old notes.
- Approach C - The Digital Marie Kondo: Switch to the new app while also converting everything old into the new system.
You can actually work through all three of these in order, A -> B -> C, until you're living 100% in the new app. That's pretty much what I'm doing with my mom and her document collection.
STEP 1: First, I'll walk through the basics of the shiny new Obsidian installation so she can get comfortable with the process. We don't need to have a perfect folder structure built out or a well-rounded collection of tags, or any of that. Step 1 is just to achieve simple notes written in a new tool. (Obsidian makes it easy to shuffle files around at any time, so there's no pressure.)
STEP 2: Second, we'll focus on using Obsidian for anything she would previously have captured in Word. This is probably the most challenging part, because it means changing a long-established habit. I don't expect it to happen overnight, but my hope is that the simpler workflow will be inherently appealing after a bit of practice, so it will turn into a pleasure rather than a chore. This stage will also highlight any cases where Word really does make sense (such as professional correspondence or material-for-print).
STEP 3: Third, to make the new notes system more useful, we'll move Word documents over into Obsidian notes a little at a time, starting with the most recent and topically relevant ones. This approach should minimize up-front work while maximizing up-front value. We may get to a point where there's no compelling reason to keep pulling in documents older than some cutoff point--maybe not even more than 6-12 months back.
HOWEVER... nerd excitement alert**
With this in mind, and since my mom is currently occupied, I decided to dive into a directly related project that I've been mulling over for months, or even years: Tidy Organizer. This is a tool specifically designed to help you organize a giant mess of files by figuring out what you have, analyzing it in context, and recommending a course of action. And, where possible, actually performing that action if you approve the recommendation.
The AI-powered tools now available to anyone technically inclined are absolutely capable of doing this, as long as you give them the right information and ask the right questions. They are excellent at classification and finding patterns in data.
Of course, I've also mentioned before that I don't really want to hand over decades of personal information (mine or my mom's) to something like ChatGPT, because I can't know what will actually happen to that data once it leaves my computer. Realistically, the answer is "probably nothing bad." But it still makes me uneasy, and it would probably make a lot of you uneasy as well.
Fortunately, the rapid advancements occurring in the AI world today keep pushing better performance into cheaper hardware. You don't need a $50k or $20k server; you can do a lot with a $2k computer that you might have bought for gaming or something anyway, or a modern MacBook. In another year, this kind of local-AI tool on your own computer will be even more accessible.
So, while the privacy-first local Tidy Organizer tool I'm showing you here (and developing while you read this) is not something most of you could use yet, it will become a viable option before too much longer. And, for those of you not too concerned with privacy, it can also be made to work with powerful cloud services like ChatGPT, Gemini, or Claude.
But I'm getting ahead of myself.
What I've thrown together so far is a tool that scans a collection of files (my mom's document collection in this case) and has local-only AI pull out a bunch of relevant information:
- Filename, location, creation, and modification date
- Who wrote or created the file
- What category of document it is (e.g. correspondence, meeting notes, invoice)
- What topics are discussed (e.g. health, diet, parenting)
- What people are mentioned
- What organizations are mentioned
- A short summary of the entire document
- Any probable related documents already known
It's currently text-only, without a pretty graphical interface, but the interface isn't the hard part--the logic is. Here's a screenshot showing the analysis of one sample file:
There's a lot that could be added to this. Right now, I'm building features around what I'm doing with my mom's document collection, while also considering how to keep general enough for others to throw at their own messy file collections at some point. Quickly reviewing individual file analysis, re-analyzing known files based on new context, figuring out which files belong together, figuring out what's relevant to current interests, and so on.
One of the more basic features already working very well is converting Word documents into the Obsidian-friendly "Markdown" format, which will make it painless to import her existing documents into the new note-taking app. (Just about any notes app these days can play nice with Markdown, so this is a great step even if Obsidian doesn't work out for some reason.) I wasn't looking forward to doing that by hand, and now I don't have to!
I've had tons of fun building this tool over the last weekend and a few evenings. While it's got a long way to go, the parts that are already done will streamline my Q4 2025 Consistency Challenge project for my mom and my efforts when I return to my own massive data archive.
While I don't assume many of you will jump at this opportunity yet, I want to ask: is Tidy Organizer something you would like access to?
- "Yes, right away, even in its unfinished state because I have my own local AI too!"
- "Yes, eventually, once it's tested and has a more friendly graphical UI."
- "Maybe, but I want to see where it goes first."
- "Probably not, because [whatever reason]."
Let me know! The code is not published yet, but I intend to open-source it. This is the kind of project people have to trust in order to use it well. And if you aren't a current usage candidate but you know someone who might like to take a look, feel free to forward this email along to them.
That's it for now. Until next week, happy data-taming!