Beanstalk Longshot test farm
Menu

#391 perf(vk): six small costs of a draw and of a submission's parts removed

by p0lux closed Open on Forgejo

Commit

Head
84a761d
Base
main 544229a
Approval
approved HumanBean, · reviewed

Plan (gpt-6.1-terra)

Compare renderer performance and pictures in Kokiri Forest and Hyrule Field

In plain words

The plan compares main and the pull request in Kokiri Forest and Hyrule Field, with a settled, stationary view in each. It takes about 15 minutes per tester and measures frame rate, frame times, draws and process CPU and memory; it does not measure replay-thread time directly or guarantee a CPU-bound view. Crashes, device loss, substantial performance regressions or large picture differences fail the plan.

Raw plan 112 lines Download plan.json

  "pass" 
    "crash" false
    "fps_mean" 
      "min_delta_pct" 15
    
    "screenshot" 
      "max_diff" 0.15
    
    "frametime_p99_ms" 
      "max_delta_pct" 20
    
    "log_forbidden_hits" 
      "max" 0
    
  
  "probe" 
    
      "kind" "crash"
    
    
      "kind" "frametime"
    
    
      "kind" "draws"
    
    
      "kind" "process"
    
    
      "kind" "log_match"
      "expected" 
      "forbidden" 
        "vk: device lost"
      
    
  
  "matrix" 
  "schema" 3
  "target" 
    
      "name" "any"
      "tags" 
      "max_reports" 2
      "min_reports" 1
    
  
  "summary" "Compare renderer performance and pictures in Kokiri Forest and Hyrule Field"
  "baseline" 
    "ref" "main"
  
  "scenario" 
    
      "env" 
      "name" "forest-and-field"
      "runs" 1
      "step" 
        
          "do" "teleport"
          "to" "KokiriForest_Child"
        
        
          "do" "wait"
          "map" "KokiriForest_Child"
          "event" "map"
          "timeout_s" 60
        
        
          "do" "measure"
          "name" "forest"
          "warmup_s" 20
          "duration_s" 60
        
        
          "do" "shot"
          "name" "forest"
          "upload" false
          "compare" "baseline"
        
        
          "do" "teleport"
          "to" "HyruleField_Child"
        
        
          "do" "wait"
          "map" "HyruleField_Child"
          "event" "map"
          "timeout_s" 60
        
        
          "do" "measure"
          "name" "field"
          "warmup_s" 20
          "duration_s" 60
        
        
          "do" "shot"
          "name" "field"
          "upload" false
          "compare" "baseline"
        
      
      "start" "new"
      "matrix" true
      "settings" 
        "vsync" 2
        "frame_rate_cap" 6
        "frame_generation" 0
      
    
  

Targets

TargetTierReportsStatus
anyreviewed HumanBean none yet 0/1 pass, 0 running, 0 queued

Reports

ReportCommitTargetTesterResultWhen
rp_618a87b57035785a14a057bany lick/agent-b0d3c1fail outlier
rp_9c6e2cce20b86ae114a057bany futondestroyer/windows-rtx4070fail outlier

Jobs

JobCommitTargetStateTesterStartedAttemptsEstimateNote
jb_ee54418b38757571 log84a761dany cancelled build-baseFlashman4/agent-de36c51 the PR was closed
jb_0996d7f40bff6cfe log84a761dany cancelled build-prthebasedpisces/agent-8d29991 the PR was closed
jb_d074271291f0bf2d log14a057bany done runfutondestroyer/windows-rtx40701
jb_8f2d3a29699284d9 log14a057bany done runlick/agent-b0d3c11