Skip to content

test flake: TestScaleTestDashboard fails intermittently #9168

Closed
@johnstcn

Description

@johnstcn

Seen in https://github.com/coder/coder/actions/runs/5900709769/job/16005875708

=== FAIL: cli TestScaleTestDashboard (1.24s)
    t.go:85: 2023-08-18 09:07:17.991 [debu]  workspaceapps.stats_collector: workspace app stats collector started
    t.go:85: 2023-08-18 09:07:17.991 [debu]  metrics_cache: deployment stats metrics refreshed  took=14.978µs  interval=5m0s
    t.go:85: 2023-08-18 09:07:18.004 [debu]  metrics_cache: template daus metrics refreshed  took=9.951814ms  interval=1h0m0s
    t.go:85: 2023-08-18 09:07:18.135 [debu]  POST  host=localhost:33771  path=/api/v2/users/first  proto=HTTP/1.1  remote_addr=127.0.0.1  took=103.606849ms  status_code=201  latency_ms=103  request_id=25ee560d-d121-459b-8253-827199074d66
    t.go:85: 2023-08-18 09:07:18.176 [debu]  POST  host=localhost:33771  path=/api/v2/users/login  proto=HTTP/1.1  remote_addr=127.0.0.1  took=37.396073ms  status_code=201  latency_ms=37  request_id=71955d77-5e72-4fa3-a0cd-3760db6e8574
    clitest.go:70: invoking command: coder --global-config /tmp/TestScaleTestDashboard20[243](https://github.com/coder/coder/actions/runs/5900709769/job/16005875708#step:5:244)90170/002 exp scaletest dashboard --count 1 --min-wait 100ms --max-wait 1s --timeout 1s --scaletest-prometheus-address 127.0.0.1:0 --scaletest-prometheus-wait 0s
    t.go:85: 2023-08-18 09:07:18.188 [debu]  cli-telemetry: report sent  request_id=14990373-4006-4cd3-9a23-314bdd39453e  count=1
    t.go:85: 2023-08-18 09:07:18.188 [debu]  GET  host=localhost:33771  path=/api/v2/entitlements  proto=HTTP/1.1  remote_addr=127.0.0.1  took=247.077µs  status_code=404  latency_ms=0  request_id=14990373-4006-4cd3-9a23-314bdd39453e
    t.go:85: 2023-08-18 09:07:18.188 [debu]  GET  host=localhost:33771  path=/api/v2/buildinfo  proto=HTTP/1.1  remote_addr=127.0.0.1  took=320.727µs  status_code=200  latency_ms=0  request_id=afcf6192-264b-4a2b-8d1e-d9960c66a332
    ptytest.go:131: 2023-08-18 09:07:18.197: cmd: "Running load test..."
    t.go:85: 2023-08-18 09:07:18.199 [debu]  GET  host=localhost:33771  path=/api/v2/users/me  proto=HTTP/1.1  remote_addr=127.0.0.1  took=10.079847ms  status_code=200  latency_ms=10  request_id=8e0db2cb-6ce1-4ada-8310-9d1329d1bab4
    t.go:85: 2023-08-18 09:07:18.202 [debu]  GET  host=localhost:33771  path=/api/v2/users/me  proto=HTTP/1.1  remote_addr=127.0.0.1  took=858.052µs  status_code=200  latency_ms=0  request_id=006ae0a9-178a-4c25-8b28-79d8c2dea6f5
    t.go:85: 2023-08-18 09:07:18.650 [debu]  GET  host=localhost:33771  path=/api/v2/workspaces  proto=HTTP/1.1  remote_addr=127.0.0.1  took=442.677772ms  status_code=200  latency_ms=442  request_id=e29e1641-69be-49bb-ba11-c3b7664e44d9
    t.go:85: 2023-08-18 09:07:19.005 [debu]  GET  host=localhost:33771  path=/api/v2/organizations/932f5f83-8944-4487-a4f4-3d3775a298e3/templates  proto=HTTP/1.1  remote_addr=127.0.0.1  took=354.134172ms  status_code=200  latency_ms=354  request_id=02399848-e9c5-42da-ac02-[244](https://github.com/coder/coder/actions/runs/5900709769/job/16005875708#step:5:245)0404283d0
    exp_scaletest_test.go:106: 
        	Error Trace:	/home/runner/actions-runner/_work/coder/coder/cli/exp_scaletest_test.go:106
        	Error:      	Received unexpected error:
        	            	running command "coder exp scaletest dashboard": load test failed, see above for more details:
        	            	    github.com/coder/coder/cli.(*RootCmd).scaletestDashboard.func1
        	            	        /home/runner/actions-runner/_work/coder/coder/cli/exp_scaletest.go:1138
        	Test:       	TestScaleTestDashboard
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: ""
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "== FAIL: dashboard/dashboard-0"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: ""
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "\tError: Get \"http://localhost:33771/api/v2/users?q=\": context deadline exceeded"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: ""
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "\tLog:"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: ""
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: ""
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "Test results:"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "\tPass:  0"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "\tFail:  1"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "\tTotal: 1"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: ""
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "\tTotal duration: 1.000712818s"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "\tAvg. duration:  1.000648888s"
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: ""
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "Uploading traces..."
    ptytest.go:131: 2023-08-18 09:07:19.190: cmd: "Waiting 0s for prometheus metrics to be scraped"
    ptytest.go:120: 2023-08-18 09:07:19.191: cmd: copy done: <nil>
    ptytest.go:121: 2023-08-18 09:07:19.191: cmd: closing out
    ptytest.go:123: 2023-08-18 09:07:19.191: cmd: closed out: <nil>
    t.go:85: 2023-08-18 09:07:19.192 [warn]  GET  host=localhost:33771  path=/api/v2/users  proto=HTTP/1.1  remote_addr=127.0.0.1  took=182.711544ms  status_code=500  latency_ms=182  request_id=d18e482e-9d39-410c-86e7-ffa574f26dd6 ...
        response_body= {
                       	"message": "Internal error fetching users.",
                       	"detail": "(dev error) prepare sql filter: new partial authorizer: prepare: eval_cancel_error: caller cancelled query execution"
                       }
    ptytest.go:95: 2023-08-18 09:07:19.197: cmd: closing expecter: PTY close
    ptytest.go:86: 2023-08-18 09:07:19.198: cmd: closing logw
    ptytest.go:88: 2023-08-18 09:07:19.198: cmd: closed logw: <nil>
    ptytest.go:86: 2023-08-18 09:07:19.198: cmd: closing logr
    ptytest.go:88: 2023-08-18 09:07:19.198: cmd: closed logr: <nil>
    ptytest.go:112: 2023-08-18 09:07:19.198: cmd: closed expecter
    t.go:85: 2023-08-18 09:07:19.198 [debu]  workspaceapps.stats_collector: flushing workspace app stats
    t.go:85: 2023-08-18 09:07:19.198 [debu]  workspaceapps.stats_collector: flushed workspace app stats
    t.go:85: 2023-08-18 09:07:19.198 [debu]  workspaceapps.stats_collector: workspace app stats collector stopped

Metadata

Metadata

Assignees

Labels

s4Internal bugs (e.g. test flakes), extreme edge cases, and bug risks

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions