conjfrnk/blackbird
Languages
A minimal, macOS-native terminal emulator.
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 | 17 / 30 | $3.55 | $5/$25 | 11m29s | |||
2 | 16 / 30 | $0.70 | $1.25/$4.25 | 5m52s | |||
3 | 15 / 30 | $0.07 | $0.2/$1.2 | 4m03s |
Key Takeaways
- Muse Spark 1.2 with Mini-SWE-agent resolves 16 of 30 tasks, one behind Claude Opus 5 and one ahead of GPT-5.6 Luna.
- GPT-5.6 Luna with Mini-SWE-agent resolves 15 of 30 tasks at $0.07 per test, the lowest supplied cost.
- Claude Opus 5 with Mini-SWE-agent reaches 17 of 30 tasks at $3.55 per test, while Muse Spark 1.2 reaches 16 at $0.70.
Model Comparison
Accuracy
56.67%
Claude Opus 5
53.33%
Muse Spark 1.2
Task outcomes
30 tasks
Cost / test
$3.55
Claude Opus 5
$0.70
Muse Spark 1.2
Cost distribution
Latency
11m 29s
Claude Opus 5
5m 52s
Muse Spark 1.2
Latency distribution
Cost Analysis
Average Token Use / Test
Cost is the clearest tradeoff in this comparison. Claude Opus 5 leads at 56.67% for $3.55 per test. Muse Spark 1.2 is the lower-cost option at 53.33% for $0.70 per test.
Latency Analysis
Average Response Time / Test
Latency separates several models with similarly strong scores. Claude Opus 5 leads at 56.67%, while GPT-5.6 Luna is fastest at 4m 3s with 50.00% accuracy.
Tasks with failures
| Models | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5 | ||||||||||||||||
| Muse Spark 1.2 | ||||||||||||||||
| GPT-5.6 Luna |
Task detail
076d3a4Issue statement
When a terminal session's persistent OSC 8 hyperlink URI cache reaches its 1 MiB intern budget, new hyperlink URIs lose attribution without leaving any diagnostic breadcrumb. Record this budget-exhaustion condition when it first occurs so support logs explain why links stopped working. The diagnostic must be emitted at most once per terminal session even if later snapshots continue to encounter new over-budget URIs; normal hyperlink interning and the existing budget behavior must remain unchanged.
View Hidden Tests
diff --git a/core/src/tests.rs b/core/src/tests.rsindex 30660d3..9fb0dc6 100644--- a/core/src/tests.rs+++ b/core/src/tests.rs@@ -10,6 +10,42 @@ fn alacritty_terminal_is_linked() { let _ = std::mem::size_of::<alacritty_terminal::term::Config>(); } +/// The global OSC 8 URI budget is a recoverable support condition, not an+/// invisible failure: the first over-budget snapshot must latch its diagnostic,+/// and later snapshots must preserve the one-shot state.+#[test]+fn osc8_intern_budget_exhaustion_is_logged_once() {+ unsafe {+ let term = bb_term_new(40, 30, 100);+ assert!(!(*term).osc8_intern_cap_logged);++ let bulk = "a".repeat(4000);+ for i in 0..300u32 {+ let uri = format!("https://example.com/{i:03}-{bulk}");+ let seq = format!("\x1b]8;;{uri}\x1b\\X\x1b]8;;\x1b\\");+ bb_term_input(term, seq.as_ptr(), seq.len());+ }++ let first = bb_term_take_snapshot(term);+ assert!(!first.is_null());+ assert!(+ (*term).osc8_intern_cap_logged,+ "crossing the URI intern budget must record that its diagnostic fired"+ );++ let second = bb_term_take_snapshot(term);+ assert!(!second.is_null());+ assert!(+ (*term).osc8_intern_cap_logged,+ "the diagnostic latch must remain set on subsequent over-budget snapshots"+ );++ bb_snap_release(first);+ bb_snap_release(second);+ bb_term_free(term);+ }+}+ #[test] fn new_and_free_roundtrip() { unsafe {