sashabaranov/go-openai
Languages
OpenAI, GPT 5.6, GPT-Image-2, Whisper API clients for Go
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 29 / 30 | $1.21 | $5/$25 | 5m17s | ||
2 | Mini-SWE-agent | 25 / 30 | $0.67 | $5/$30 | 3m24s | ||
3 | Mini-SWE-agent | 25 / 30 | $0.15 | $1.4/$4.4 | 5m30s | ||
4 | Mini-SWE-agent | 24 / 30 | $0.01 | $0.44/$1.32 | 63.38s | ||
5 | Mini-SWE-agent | 23 / 30 | $0.04 | $0.2/$1.2 | 3m17s |
Key Takeaways
- Claude Opus 5 with Mini-SWE-agent scores 96.67% at $1.21 per test; this 30-task result is directional.
- GLM 5.2 (Fireworks) and GPT-5.6 Sol with Mini-SWE-agent tie at 25 of 30; GPT-5.6 Sol is faster, while GLM 5.2 (Fireworks) costs less.
- Deepseek V4 Flash 0731 with Mini-SWE-agent resolves 24 of 30 at $0.01 per test, ahead of GPT-5.6 Luna’s 23 at $0.04.
Model Comparison
Accuracy
96.67%
Claude Opus 5
83.33%
GLM 5.2
Task outcomes
30 tasks
Cost / test
$1.21
Claude Opus 5
$0.15
GLM 5.2
Cost distribution
Latency
5m 17s
Claude Opus 5
5m 30s
GLM 5.2
Latency distribution
Cost Analysis
Average Token Use / Test
Cost is the clearest tradeoff in this comparison. Claude Opus 5 leads at 96.67% for $1.21 per test. DeepSeek V4 Flash 0731 is the lower-cost option at 80.00% for $0.01 per test.
Latency Analysis
Average Response Time / Test
Latency separates several models with similarly strong scores. Claude Opus 5 leads at 96.67%, while DeepSeek V4 Flash 0731 is fastest at 1m 3s with 80.00% accuracy.
Tasks with failures
| Models | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5 | |||||||||
| GLM 5.2 | |||||||||
| GPT-5.6 Sol | |||||||||
| DeepSeek V4 Flash 0731 | |||||||||
| GPT-5.6 Luna |
Task detail
3d58ac5Issue statement
Add transcription chunking strategy support to audio requests. AudioRequest must allow callers to select automatic chunking with the string auto or provide a structured server-VAD configuration containing a type, prefix padding in milliseconds, silence duration in milliseconds, and threshold. When set, the strategy must be sent as the chunking_strategy multipart form field: strings are sent verbatim and structured values are JSON encoded. When unset, the field must be omitted. Errors produced while encoding or writing this field should be returned to the caller.
View Hidden Tests
diff --git a/audio_chunking_strategy_hidden_test.go b/audio_chunking_strategy_hidden_test.gonew file mode 100644index 0000000..6ac6380--- /dev/null+++ b/audio_chunking_strategy_hidden_test.go@@ -0,0 +1,71 @@+package openai++import (+ "os"+ "path/filepath"+ "reflect"+ "testing"++ "github.com/sashabaranov/go-openai/internal/test"+)++func requestWithChunkingStrategy(t *testing.T, path string, strategy any) AudioRequest {+ t.Helper()+ req := AudioRequest{FilePath: path, Model: GPT4oTranscribeDiarize}+ value := reflect.ValueOf(&req).Elem()+ field := value.FieldByName("ChunkingStrategy")+ if !field.IsValid() {+ t.Fatalf("AudioRequest does not expose ChunkingStrategy")+ }+ if !field.CanSet() || !reflect.TypeOf(strategy).AssignableTo(field.Type()) {+ t.Fatalf("AudioRequest.ChunkingStrategy cannot accept %T", strategy)+ }+ field.Set(reflect.ValueOf(strategy))+ return req+}++func TestAudioChunkingStrategyMultipartEncoding(t *testing.T) {+ path := filepath.Join(t.TempDir(), "fake.mp3")+ test.CreateTestFile(t, path)++ tests := []struct {+ name string+ strategy any+ want string+ }{+ {name: "automatic", strategy: "auto", want: "auto"},+ {name: "server VAD configuration", strategy: struct {+ Type string `json:"type"`+ PrefixPaddingMs int `json:"prefix_padding_ms,omitempty"`+ SilenceDurationMs int `json:"silence_duration_ms,omitempty"`+ Threshold float64 `json:"threshold,omitempty"`+ }{"server_vad", 300, 200, 0.5}, want: `{"type":"server_vad","prefix_padding_ms":300,"silence_duration_ms":200,"threshold":0.5}`},+ }++ for _, tt := range tests {+ t.Run(tt.name, func(t *testing.T) {+ var got string+ seen := false+ builder := &mockFormBuilder{+ mockCreateFormFile: func(string, *os.File) error { return nil },+ mockWriteField: func(name, value string) error {+ if name == "chunking_strategy" {+ seen, got = true, value+ }+ return nil+ },+ mockClose: func() error { return nil },+ }+ req := requestWithChunkingStrategy(t, path, tt.strategy)+ if err := audioMultipartForm(req, builder); err != nil {+ t.Fatalf("audioMultipartForm returned error: %v", err)+ }+ if !seen {+ t.Fatal("chunking_strategy multipart field was not written")+ }+ if got != tt.want {+ t.Fatalf("chunking_strategy = %q, want %q", got, tt.want)+ }+ })+ }+}