pythondashboard

Solving search for developers

Search has become difficult for developers. I am proposing a simple solution that has been working for me for a few years now. The main realization was that developer searches are mostly keyword-based. That brings us a simple solution that uses bookmarks.

The plan

Here's the plan in full:

  1. Share bookmarks through GitHub
  2. Process and categorize the entries
  3. Make them available as Google Chrome bookmarks

Why it works

As mentioned above, the process has been working me for a few reasons:

  1. Our search queries are mostly keyword-based
  2. We search for the same thing over and over
  3. We reuse old search queries and seldom come up with new ones
  4. We find answers in issue trackers, message boards, & articles
  5. Google Chrome URL bar & local bookmarks are perfect for search

Search prefixes

Let's expand on item 5 for a minute. I have about 5k entries in my Other bookmarks folder. They are synced across all my devices. And I have been using keyword prefixes for my entries. In fact, having search queries in mind when adding a new entry helps a lot. Here are some keyword prefixes I use:

I tend to combine prefixes. For example, article might be followed by a programming language and other prefixes related to the topic:

An entry might be: article golang caching gocache: ...

So, when to bookmark?

I bookmark all the time, especially on my phone. I would often go to the trending pages on GitHub to add new entries to my bookmarks. But I tend to write repository names after the prefix, skipping organizations and account names. For example: github django. I also lowercase all my entries. It's a small detail but one that pays great dividends when using a mobile device. Typing mix-case is very inefficient on mobile devices.

Well, bookmark everything?

Certainly not. I have, for instance, gave up the prefixes doc and docs. My experience has been that documentations are live documents. They redirect to new versions, get deprecated, exist in various forms & formats, live at multiple locations, etc. They aren't worth the trouble.

Speaking of redirects, here's the biggest problem I have with this solution so far. Last time I went through my bookmarks to check on my github entries. I noticed that most repositories had been deprecated, abandoned, or deleted. The manual pruning process was painful and annoying.

Will this scale?

I suspect that, locally, it won't. I have about 5k bookmark entries. Suppose that 10k developers having roughly the same number of entries contributed to the repository. That's about 50m entries, or maybe 30m to be more conservative. I suspect that Google Chrome won't be able to handle it. But I would love to see them impress us! Let's also keep in mind that the search front-end could be external to Google Chrome, with a more appropriate search back-end, etc.

What about pocket.com, feedly.com, pinboard.in, etc?

They seem not effective for search. Not for me at least. I do, in fact, have a pocket.com and other accounts that I have not opened for some time. There is that feeling of pressure with anything you put on them that you have to come back to and read. Rather than having a place you can go to search for something.

January 23rd, 2020.