In the last post in the series on algorithmic feeds in Ktistec, I introduced the built-in criteria judge, which renders verdicts on posts based on the presence or absence of terms, like hashtags and keywords.

In this post, I introduce custom algorithmic feeds by showing you what a simple implementation of the criteria judge looks like if rewritten as a custom judge. Custom judges are nothing more than scripts, external to the server, that the server runs to judge candidate posts.

Here is the custom criteria judge:

#!/usr/bin/env python3
# ktistec: batch-size=250
# ktistec: param=any required
# ktistec: param=all
# ktistec: param=none
import html, json, re, sqlite3, sys
def terms(value):
return [t.strip().lower() for t in (value or "").split(",") if t.strip()]
db = sqlite3.connect(sys.argv[1])
params = json.loads(db.execute("SELECT params FROM feed").fetchone()[0])
any_terms = terms(params.get("any"))
all_terms = terms(params.get("all"))
none_terms = terms(params.get("none"))
posts = db.execute("SELECT iri, name, summary, content, hashtags, mentions, labels FROM posts")
for iri, name, summary, content, hashtags, mentions, labels in posts.fetchall():
text = " ".join(filter(None, (content, summary, name)))
text = html.unescape(re.sub(r"<[^>]+>", " ", text)).lower()
hashtags = {h["name"].lower() for h in json.loads(hashtags)}
mentions = {m["name"].lower() for m in json.loads(mentions)}
labels = {label.lower() for label in json.loads(labels)}
def hit(term):
if term.startswith("#"):
return term[1:] in hashtags
if term.startswith("@"):
return term[1:] in mentions
if term.startswith("§"):
return term[1:] in labels
return term in text
included = not any(map(hit, none_terms)) and any(map(hit, any_terms)) and all(map(hit, all_terms))
db.execute("INSERT INTO answers (iri, included) VALUES (?, ?)", (iri, int(included)))
db.commit()

It's simple, and it's not a perfect work-alike, but it shows the overall structure.

The Ktistec server runs judges, both built-in and custom, when there are candidate posts. Given a batch of posts, the judge makes exactly one decision: is each post included in the feed or excluded from the feed?

To do this, a judge is given the path to a temporary SQLite file (in sys.argv[1] in the example above) with information about the posts in the batch and their authors, as well as information about the feed itself.

Most of the time, posts arrive one at a time and the script runs on each post as it arrives. But the SQLite database can contain multiple posts—and it will when previewing or backfilling a feed—so the script must iterate over every post and judge each post independently.

In this example, the judge strips the post of markup and then looks for terms that match. It writes a verdict on the post into the answers table.

The front matter at the top of the file tells the server some things about the judge: the maximum batch size (the default is 1), and what params the judge expects the feed to provide. In this case, it expects three params, and the first is required. The params allow a user to create multiple, parameterized feeds from a single judge.

The params appear as inputs on the feed editor page.

The "Judge" dropdown at the top of the form only appears when you create a feed, and only when there are custom judges. You make a script a custom judge by placing the script in the <root>/etc/judges directory, making it executable, and waiting a few minutes until the server picks it up, or restarting the server. Here the custom criteria judge was saved as <root>/etc/judges/criteria-alike.

The example is in Python, but you can use the language of your choice.

In the next post, I'll go over what else in the SQLite database supplied to the judge is useful for judging.

#ktistec #AlgorithmicFeed #ActivityPub #fediverse

Since v3.13.0, Ktistec has supported custom feeds.

A custom feed is a collection of posts selected by a set of rules called a judge. Any post Ktistec has cached that is public, or was sent to you, is a candidate. The judge's job is to decide which posts are in the feed and which posts are out.

Ktistec comes with one built-in judge called the criteria judge. It also supports any number of fully custom external judges—judges implemented as external scripts the server can call on candidate posts.

I use the criteria judge for most of my custom feeds. This post explains how it works. Later posts will dive into external judges.

The Criteria Judge

The criteria judge allows you to define feeds by flexibly matching on keywords, hashtags, and mentions. You can filter candidates to those posts that contain any of one set of keywords, hashtags, or mentions, all of a second set, and none of a third.

Keywords match anywhere in a post's text, summary, or title, ignoring case—even inside words, so "art" also matches "party". Hashtags and mentions match exactly: #art matches only posts tagged #art. Use a hashtag when you want precision. Use a keyword when you want reach.

Once you've settled on terms, you can preview the feed. To keep it fast, the preview returns up to 20 matches from the 100 most recent candidates—just enough to give you a feel for what it will look like. "Create Feed" publishes the feed and starts a task that backfills the last 30 days of posts. (Feeds are private, though—only you can see them.)

After it is published, the feed is available in the deck—the column-oriented feed reader. The deck shows up to 12 feeds side by side, and you can reorder them.

Matching is fast. I have feeds with 20+ keywords and hashtags, and there is no perceptible processing delay when a new post arrives.

#ktistec #activitypub #fediverse