Discussion

In July 2024 I sat in a training room in Nusa Dua with a laptop and too many DNS diagrams. PANDI, the...#security #ai #dns #software #coding #development #engineering #inclusive #community
From DNS Security Trainer to ICANN87 Fellow: What a Cybersecurity Engineer Looks for in Bali

Can any of these #AI #leaders give a straightforward answer about the future of #jobs ?

youtube.com/shorts/SHudc...

Can any of these AI leaders gi...

Centralisering internet geeft poortwachters Big Tech teveel macht.
4 lessen na 20 jaar Twitter: https://www.groene.nl/artikel/op-zoek-naar-het-nieuwe-dorpsplein
1⃣ Klein voelt fijn
2⃣ Vrije meningsuiting valt niet te privatiseren
3⃣ Maak van algoritmes geen manipulatie-machine
4⃣ Geef de gebruiker de macht
#socialmedia #twitter #VvMU #freespeech #AI #algoritme
https://mastodon.social/@Matthijs85/116793565805378214

Making that failure case more graceful and documenting what people should be doing instead took 650 fewer lines than Claude generated.

Australian underwater #archaeology reveals landscape and cultural heritage of great significance on the seabed.

"The Ngarda-Ngarli people have cared for Murujuga for at least 50,000 years. What this research shows is that our cultural landscape extends far beyond the shoreline and into Sea Country, where important parts of our story remain preserved beneath the sea,"

https://phys.org/news/2026-10-australia-oldest-underwater-archaeology-reveals.html

@clockssarchive.bsky.social argues that "AI companies have a stake in the continued existence of dark archives" (as well as open ones).
clockss.org/open-access-...

#AI #CLOCKSS #OpenAccess #Preservation

Turn nine of a support chat feels fine. Turn ninety is a different animal. Every request resends the...#dotnet #csharp #ai #chatgpt #software #coding #development #engineering #inclusive #community
Your Chat History Is Eating the Context Window — Fix It With a Reducer

Nasdaq futures fall 1% as #AI rout deepens on debt concerns - #stocks www.reuters.com/business/nas...

Nasdaq futures fall 1% as AI r...

A rate limit is not a failure, so do not spend retry attempts on it. Work out which limit you hit,...#ai #node #redis #backend #software #coding #development #engineering #inclusive #community
Do Not Retry an LLM Rate Limit, Reschedule It

A single failing golden case tells you almost nothing, and a single passing run tells you even less. If your grader is a string match, a judge model, or a schema check on free-form output, then the same prompt can pass six times and fail twice without anything in your system changing. The practical fix is not a heavier eval framework; it is a per-case noise floor measured from repeated baseline runs, plus a rule that refuses to escalate a verdict the data cannot support.

This article builds that gate in roughly eighty lines of standard-library Python. It assumes you already keep a stamp of model, sampling parameters, and grader version next to every score, because a score without its stamp is not comparable to anything. The artifact here is the layer above that: repetition, a one-sided Wilson bound, and a quarantine list for cases that are too unstable to judge at all.

Why one run cannot separate noise from a regression

Think of each graded case as a coin whose bias you are trying to estimate from a handful of flips. A prompt that passes five of eight baseline repeats and three of eight candidate repeats looks alarming on a dashboard, but with n=8 the confidence interval is simply too wide to justify that alarm. Working the numbers makes the point concrete: six passes out of eight gives a one-sided 95% Wilson lower bound near 0.46, while three out of eight gives roughly 0.16. The candidate is below the baseline floor, yet it is not zero, so the honest verdict is "inconclusive, sample more" rather than "regression".

That distinction is where most eval suites quietly lose their credibility. Teams wire a threshold to a single run, the threshold fires on sampling jitter, and within a month everyone has learned to ignore the report. A noise floor turns the report back into a signal, because it encodes how much variation the harness itself produces before any prompt changes.

The harness: repetition, a floor, and a stamp guard

The module below is runnable as written; you only supply the load_project() adapter that returns your cases plus generate and grade callables.

#!/usr/bin/env python3
"""eval_noise_floor.py - separate grader noise from a real prompt regression.

Stdlib only, Python 3.10+. You supply load_project().
"""
from __future__ import annotations

import argparse, hashlib, json, math, pathlib, sys
from dataclasses import dataclass, asdict

Z = 1.645 # one-sided 95%
FLAKE_BAND = 0.25 # baseline-vs-baseline gap that marks a case unstable
SLACK = 0.05 # jitter you tolerate without escalating

def wilson_lower(passes: int, n: int, z: float = Z) -> float:
"""Lower bound of the Wilson score interval for a binomial proportion."""
if n == 0:
return 0.0
p = passes / n
denom = 1.0 + z * z / n
centre = p + z * z / (2 * n)
margin = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n))
return max(0.0, (centre - margin) / denom)

@dataclass
class CaseResult:
case_id: str
passes: int
n: int

@property
def rate(self) -> float:
return self.passes / self.n if self.n else 0.0

def stamp(model_id, temperature, top_p, grader_version, case_ids) -> str:
payload = json.dumps(
{"model": model_id, "temperature": temperature, "top_p": top_p,
"grader": grader_version, "cases": sorted(case_ids)},
sort_keys=True,
)
return hashlib.sha256(payload.encode()).hexdigest()[:12]

def run(cases, generate, grade, repeats: int) -> list[CaseResult]:
out =
[] for case in cases:
passes = 0
for i in range(repeats):
completion = generate(case["prompt"], i)
passes += int(bool(grade(case, completion)))
out.append(CaseResult(case["id"], passes, repeats))
return out

def verdict(base_a, base_b, cand) -> str:
if abs(base_a.rate - base_b.rate) > FLAKE_BAND:
return "quarantine"
floor = max(0.0, wilson_lower(base_a.passes, base_a.n) - SLACK)
if cand.rate >= floor:
return "within_noise"
if cand.passes == 0:
return "regressed"
return "inconclusive"

def main(argv=None) -> int:
ap = argparse.ArgumentParser()
sub = ap.add_subparsers(dest="cmd", required=True)
c = sub.add_parser("collect")
c.add_argument("--out", required=True)
c.add_argument("--repeats", type=int, default=8)
c.add_argument("--model", required=True)
c.add_argument("--temperature", type=float, default=0.7)
c.add_argument("--grader-version", required=True)
p = sub.add_parser("compare")
p.add_argument("baseline_a")
p.add_argument("baseline_b")
p.add_argument("candidate")
args = ap.parse_args(argv)

if args.cmd == "collect":
cases, generate, grade = load_project()
results = run(cases, generate, grade, args.repeats)
manifest = {
"stamp": stamp(args.model, args.temperature, 1.0,
args.grader_version, [c["id"] for c in cases]),
"model": args.model,
"grader_version": args.grader_version,
"results": [asdict(r) | {"rate": round(r.rate, 3)} for r in results],
}
pathlib.Path(args.out).write_text(json.dumps(manifest, indent=2))
return 0

a, b, cand = (json.loads(pathlib.Path(f).read_text())
for f in (args.baseline_a, args.baseline_b, args.candidate))
if len({r["stamp"] for r in (a, b, cand)}) != 1:
print("refusing to compare: stamps differ", file=sys.stderr)
return 2
idx = [{r["case_id"]: CaseResult(r["case_id"], r["passes"], r["n"])
for r in run_["results"]} for run_ in (a, b, cand)]
table = {cid: verdict(idx[0][cid], idx[1][cid], idx[2][cid]) for cid in sorted(idx[0])}
print(json.dumps(table, indent=2))
return 1 if "regressed" in table.values() else 0

if __name__ == "__main__":
raise SystemExit(main())

The two baseline collections are the part people skip, and they are the part that pays for itself. Both are produced by the same stamp against unchanged prompts, so any gap between them is pure harness variance, which is exactly the quantity you want to know before blaming a prompt. The exit code stays non-zero only when a case fails at zero passes or after escalation, so a noisy suite stops blocking deploys it should never have blocked.

python eval_noise_floor.py collect --out runs/base_a.json --repeats 8 \
--model your-model-id --grader-version g7
python eval_noise_floor.py collect --out runs/base_b.json --repeats 8 \
--model your-model-id --grader-version g7
python eval_noise_floor.py collect --out runs/cand.json --repeats 8 \
--model your-model-id --grader-version g7
python eval_noise_floor.py compare runs/base_a.json runs/base_b.json runs/cand.json

Repetition is cheap to design and expensive to run

Repeats multiply spend linearly, and the arithmetic gets uncomfortable fast. Forty cases at k=8 for two baselines plus a candidate is 960 generations, and escalating every inconclusive case to k=16 pushes the total past two thousand calls before you have tuned anything. That cost profile is the main reason teams keep shipping one-shot evals: the honest version is roughly ten times the traffic of the dishonest one.

This is the one place where MonkeyCode's operator-supplied availability claims become relevant to the method. The project advertises free model access for prototyping, currently described as a ten-million-token allowance, and a free server option for running long repeat jobs off your laptop. Both figures and terms are promotional and time-sensitive, so confirm the current numbers on the project's own page before you size a repeat budget around them. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Keep the harness behind your own adapter regardless of which endpoint fills the generate callable. A regression gate that only runs on one provider's promotional tier is not a gate; it is a dependency, and it will fail the week the terms change.

What each verdict should trigger

Quarantined cases deserve a sentence of their own, because they are usually misread as model problems. A judge model that flips on the same completion twice, or a regex that depends on whitespace, will look exactly like an unstable prompt until you measure it twice. Pull those cases out of the gate, fix the grader, then reintroduce them with their own floor.

Limitations and who should not use this

Deterministic graders on structured output rarely need k=8, since exact key matching is effectively noiseless and a single sanity repeat is enough. Stateful agents that mutate a database, a queue, or a sandbox between calls break the independence assumption outright, so reruns must start from a reset fixture or the floor means nothing. Anyone unable to pin temperature, top-p, and grader version should stop here, because repetition without a stable stamp just produces a more expensive kind of guessing.

Regulated environments with data-retention obligations should also read the terms of any free tier before sending golden cases through it, and this harness gives you no protection there. Finally, teams chasing latency or cost budgets need a separate harness, because a pass-rate floor says nothing about the tail behaviour you actually care about.

If the repeat-heavy version sounds worth trying and the token bill is the blocker, the free model access and free server option in MonkeyCode are a reasonable place to start. Keep the adapter boundary intact, verify the terms yourself, and the gate stays yours even if the promotion does not.#testing #ai #python #llm #software #coding #development #engineering #inclusive #community
Compute a Noise Floor Before You Call an Eval Regression

Factory workers in India are being asked to wear GoPro cameras to record the way they do their jobs. The ultimate goal is to train robots to take over. Will it work?

#wellness #automation #ai

These workers help train robot...

(diesec.com) Exploitation of Unpatched AhsayCBS Backup Server Vulnerabilities Enables Remote Code Execution and Cryptocurrency Mining

In brief - This article discusses the exploitation of two vulnerabilities in AhsayCBS backup servers that allow unauthenticated remote code execution, noting that initial patches were ineffective.

Technically - This article details a vulnerability chain consisting of CVE-2026-105133 (improper authentication in checkSysPwd) and CVE-2026-105134 (OS command injection via /rps/api/json/UpdateReceivers.do). Attackers leverage an authentication bypass using random tokens to execute code as NT AUTHORITY\SYSTEM, subsequently deploying JSP web shells and a Monero miner (edge.exe) managed by a modified NSSM service (msedge.exe) under the name MicrosoftEdgeUpdateSvc. Mitigation requires installing hotfix 10.3.4.45 for version 10 or 9.15.2.70 for versions 7-9, as version 10.3.4.0 was found to be insufficient.

Source: https://diesec.com/2026/10/ahsaycbs-backup-server-vulnerability-cve-2026-105134/

#Cybersecurity

@hipsterelectron @hiker @smallcircles to solve the problem of multiple standards Fighting for adoption, we invented one more standard. Some people didn't like this and now there is another body that publishes standards.

A rotund little cat joined Nao-san for tea and to catch the last rays of the day🌞🙌🐈🍊

#哲学の道 #kyotocats #京都 #Japan #Kyoto #ThePathofPhilosophy

Originally published on dbln.me. A couple of months ago I started developing Kept, an open-source...#ai #llm #agents #architecture #software #coding #development #engineering #inclusive #community
Reliability at the core: five ground rules for building an agent loop

Originally published on The Daily Flare. Key takeaways ✓ Gemini 4 Argon beats Claude...#ai #llm #machinelearning #claude #software #coding #development #engineering #inclusive #community
Gemini 4 Argon vs Claude Opus 5.5: Benchmark Comparison

Me and Ilhan Omar are far apart policy wise. I’m pro life, she’s pro abortion. I’m pro 2A, she’s pro gun control. And on and on.

But can someone tell me how pretending to get ready to deport Ilhan Omar is going to bring gas down under $2, make health insurance freaking affordable, make a family’s weekly grocery bill make sense, and stop sending our beloved military off to fight for another fucking foreign war???

People are so sick and tired of the bullshit.

When you dumb mf’ers at the WH actually release the Epstein files and lock up the pedophiles and rapists you all hang out with, then we will start paying attention.

Until then 🖕.