Max level shown:
1 I260129 21:35:23.525477 1 util/log/file_sync_buffer.go:238 ⋮ [config] file created at: 2026/01/29 21:35:23
2 I260129 21:35:23.525492 1 util/log/file_sync_buffer.go:238 ⋮ [config] running on machine: ‹bmat-EVT22200007-0000db45›
3 I260129 21:35:23.525502 1 util/log/file_sync_buffer.go:238 ⋮ [config] binary: CockroachDB OSS v22.1.22-46-g367bca413b (x86_64-pc-solaris2.11, built 2025/10/08 03:53:50, go1.22.11)
4 I260129 21:35:23.525508 1 util/log/file_sync_buffer.go:238 ⋮ [config] arguments: [‹cockroach› ‹start-single-node› ‹--insecure› ‹--http-addr=:0› ‹--store=path=/var/tmp/omicron_tmp/.tmpzonC67/data,ballast-size=0› ‹--listen-addr› ‹[::1]:0› ‹--listening-url-file› ‹/var/tmp/omicron_tmp/.tmpzonC67/listen-url›]
5 I260129 21:35:23.525524 1 util/log/file_sync_buffer.go:238 ⋮ [config] log format (utf8=✓): crdb-v2
6 I260129 21:35:23.525530 1 util/log/file_sync_buffer.go:238 ⋮ [config] line format: [IWEF]yymmdd hh:mm:ss.uuuuuu goid [chan@]file:line redactionmark \[tags\] [counter] msg
7 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 ALL SECURITY CONTROLS HAVE BEEN DISABLED!
8 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 +
9 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 +This mode is intended for non-production testing only.
10 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 +
11 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 +In this mode:
12 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 +- Your cluster is open to any client that can access ‹::1›.
13 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 +- Intruders with access to your machine or network can observe client-server traffic.
14 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 +- Intruders can log in without password and read or write any data in the cluster.
15 W260129 21:35:23.524738 1 1@cli/start.go:1186 ⋮ [n?] 1 +- Intruders can consume all your server's resources and cause unavailability.
16 I260129 21:35:23.525914 1 1@cli/start.go:1196 ⋮ [n?] 2 To start a secure server without mandating TLS for clients,
17 I260129 21:35:23.525914 1 1@cli/start.go:1196 ⋮ [n?] 2 +consider --accept-sql-without-tls instead. For other options, see:
18 I260129 21:35:23.525914 1 1@cli/start.go:1196 ⋮ [n?] 2 +
19 I260129 21:35:23.525914 1 1@cli/start.go:1196 ⋮ [n?] 2 +- ‹https://go.crdb.dev/issue-v/53404/v22.1›
20 I260129 21:35:23.525914 1 1@cli/start.go:1196 ⋮ [n?] 2 +- https://www.cockroachlabs.com/docs/v22.1/secure-a-cluster.html
21 W260129 21:35:23.526036 1 1@cli/start.go:1112 ⋮ [n?] 3 ‹Using the default setting for --cache (128 MiB).›
22 W260129 21:35:23.526036 1 1@cli/start.go:1112 ⋮ [n?] 3 +‹ A significantly larger value is usually needed for good performance.›
23 W260129 21:35:23.526036 1 1@cli/start.go:1112 ⋮ [n?] 3 +‹ If you have a dedicated server a reasonable setting is 25% of physical memory.›
24 I260129 21:35:23.526099 1 1@cli/start.go:1225 ⋮ [n?] 4 ‹CockroachDB OSS v22.1.22-46-g367bca413b (x86_64-pc-solaris2.11, built 2025/10/08 03:53:50, go1.22.11)›
25 I260129 21:35:23.526929 1 server/config.go:485 ⋮ [n?] 5 unable to retrieve system total memory: ‹not implemented on illumos›
26 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 server configuration:
27 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 +‹max offset 500000000›
28 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 +‹cache size 128 MiB›
29 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 +‹SQL memory pool size 128 MiB›
30 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 +‹scan interval 10m0s›
31 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 +‹scan min idle time 10ms›
32 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 +‹scan max idle time 1s›
33 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 +‹event log enabled true›
34 I260129 21:35:23.527053 1 server/config.go:489 ⋮ [n?] 6 +‹span configs enabled true›
35 I260129 21:35:23.527170 1 1@cli/start.go:1090 ⋮ [n?] 7 using local environment variables:
36 I260129 21:35:23.527170 1 1@cli/start.go:1090 ⋮ [n?] 7 +GOTRACEBACK=crash
37 I260129 21:35:23.527170 1 1@cli/start.go:1090 ⋮ [n?] 7 +LANG=‹en_US.UTF-8›
38 I260129 21:35:23.527170 1 1@cli/start.go:1090 ⋮ [n?] 7 +LC_ALL=‹en_US.UTF-8›
39 I260129 21:35:23.527170 1 1@cli/start.go:1090 ⋮ [n?] 7 +TZ=‹UTC›
40 I260129 21:35:23.527252 1 1@cli/start.go:1097 ⋮ [n?] 8 process identity: ‹uid 12345 euid 12345 gid 12345 egid 12345›
41 W260129 21:35:23.527761 1 1@cli/start.go:1310 ⋮ [n?] 9 could not initialize GEOS - spatial functions may not be available: geos: error during GEOS init: geos: cannot load GEOS from dir ‹"/usr/local/lib/cockroach"›: ‹geos error: ld.so.1: cockroach: fatal: /usr/local/lib/cockroach/libgeos.so: open failed: No such file or directory›
42 I260129 21:35:23.527845 1 1@cli/start.go:581 ⋮ [n?] 10 starting cockroach node
43 I260129 21:35:23.848097 86 server/config.go:665 ⋮ [n?] 11 1 storage engine‹› initialized
44 I260129 21:35:23.848203 86 server/config.go:668 ⋮ [n?] 12 Pebble cache size: 128 MiB
45 I260129 21:35:23.848228 86 server/config.go:668 ⋮ [n?] 13 store 0: max size 0 B, max open file limit 60536
46 I260129 21:35:23.848248 86 server/config.go:668 ⋮ [n?] 14 store 0: {Encrypted:false ReadOnly:false FileStoreProperties:{path=‹/var/tmp/omicron_tmp/.tmpzonC67/data›, fs=zfs, blkdev=‹rpool/work›, mnt=‹/work› opts=‹rw,devices,setuid,nonbmand,exec,xattr,atime,dev=4350006›}}
47 E260129 21:35:23.848829 86 1@server/status/runtime.go:343 ⋮ [n?] 15 could not get initial disk IO counters: ‹not implemented yet›
48 E260129 21:35:23.848944 86 1@server/status/runtime.go:347 ⋮ [n?] 16 could not get initial disk IO counters: ‹not implemented yet›
49 I260129 21:35:23.852466 86 server/tracedumper/tracedumper.go:120 ⋮ [n?] 17 writing job trace dumps to ‹/var/tmp/omicron_tmp/.tmpzonC67/data/logs/inflight_trace_dump›
50 I260129 21:35:24.003703 86 1@server/clock_monotonicity.go:65 ⋮ [n?] 18 monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
51 I260129 21:35:24.007077 86 1@server/clock_monotonicity.go:140 ⋮ [n1] 19 Sleeping till wall time 1769722524007071393 to catches up to 1769722524503594500 to ensure monotonicity. Delta: 496.523107ms
52 I260129 21:35:24.007190 136 1@server/server.go:1319 ⋮ [n1] 20 connecting to gossip network to verify cluster ID ‹"7ccbaf7e-e8bb-4a15-9a31-b9fcf76898ed"›
53 I260129 21:35:24.504649 86 1@cli/start.go:521 ⋮ [n1] 21 listening URL file: ‹/var/tmp/omicron_tmp/.tmpzonC67/listen-url›
54 W260129 21:35:24.505110 86 1@gossip/gossip.go:1531 ⋮ [n1] 22 no addresses found; use --join to specify a connected node
55 I260129 21:35:24.505212 86 gossip/gossip.go:401 ⋮ [n1] 23 NodeDescriptor set to ‹node_id:1 address:<network_field:"tcp" address_field:"[::1]:40349" > attrs:<> locality:<> ServerVersion:<major_val:22 minor_val:1 patch:0 internal:0 > build_tag:"v22.1.22-46-g367bca413b" started_at:1769722524505201799 cluster_name:"" sql_address:<network_field:"tcp" address_field:"[::1]:40349" > http_address:<network_field:"tcp" address_field:"[::1]:53706" >›
56 I260129 21:35:24.530660 86 server/node.go:470 ⋮ [n1] 24 initialized store s1
57 I260129 21:35:24.530794 86 kv/kvserver/stores.go:250 ⋮ [n1] 25 read 0 node addresses from persistent storage
58 I260129 21:35:24.531028 86 server/node.go:549 ⋮ [n1] 26 started with engine type ‹2›
59 I260129 21:35:24.531062 86 server/node.go:551 ⋮ [n1] 27 started with attributes []
60 I260129 21:35:24.531271 86 server/goroutinedumper/goroutinedumper.go:122 ⋮ [n1] 28 writing goroutine dumps to ‹/var/tmp/omicron_tmp/.tmpzonC67/data/logs/goroutine_dump›
61 I260129 21:35:24.531332 86 server/heapprofiler/heapprofiler.go:49 ⋮ [n1] 29 writing go heap profiles to ‹/var/tmp/omicron_tmp/.tmpzonC67/data/logs/heap_profiler› at least every 1h0m0s
62 I260129 21:35:24.531355 86 server/heapprofiler/cgoprofiler.go:53 ⋮ [n1] 30 to enable jmalloc profiling: "export MALLOC_CONF=prof:true" or "ln -s prof:true /etc/malloc.conf"
63 I260129 21:35:24.531366 86 server/heapprofiler/statsprofiler.go:54 ⋮ [n1] 31 writing memory stats to ‹/var/tmp/omicron_tmp/.tmpzonC67/data/logs/heap_profiler› at last every 1h0m0s
64 W260129 21:35:24.531428 86 server/env_sampler.go:121 ⋮ [n1] 32 failed to start query profiler worker: failed to read ‹memory› cgroup from cgroups file: ‹/proc/self/cgroup›: open ‹/proc/self/cgroup›: no such file or directory
65 I260129 21:35:24.531587 86 1@server/server.go:1458 ⋮ [n1] 33 starting http server at ‹[::1]:53706› (use: ‹[::1]:53706›)
66 I260129 21:35:24.531612 86 1@server/server.go:1465 ⋮ [n1] 34 starting grpc/postgres server at ‹[::1]:40349›
67 I260129 21:35:24.531630 86 1@server/server.go:1466 ⋮ [n1] 35 advertising CockroachDB node at ‹[::1]:40349›
68 I260129 21:35:24.531636 136 1@server/server.go:1322 ⋮ [n1] 36 node connected via gossip
69 E260129 21:35:24.531680 86 server/status/recorder.go:420 ⋮ [n1,summaries] 37 could not get total system memory: ‹not implemented on illumos›
70 E260129 21:35:24.580856 282 kv/kvserver/queue.go:1106 ⋮ [n1,replicate,s1,r26/1:‹/NamespaceTable/{30-Max}›] 38 1 matching stores are currently throttled: [‹s1: draining›]
71 E260129 21:35:24.581612 62 kv/kvserver/queue.go:1106 ⋮ [n1,replicate,s1,r3/1:‹/System/{NodeLive…-tsd}›] 39 1 matching stores are currently throttled: [‹s1: draining›]
72 E260129 21:35:24.581804 160 kv/kvserver/queue.go:1106 ⋮ [n1,replicate,s1,r1/1:‹/{Min-System/NodeL…}›] 40 1 matching stores are currently throttled: [‹s1: draining›]
73 E260129 21:35:24.581866 160 kv/kvserver/queue.go:1106 ⋮ [n1,replicate,s1,r2/1:‹/System/NodeLiveness{-Max}›] 41 1 matching stores are currently throttled: [‹s1: draining›]
74 I260129 21:35:24.581878 160 kv/kvserver/queue.go:1206 ⋮ [n1,replicate] 42 purgatory is now empty
75 I260129 21:35:24.583084 286 kv/kvclient/rangefeed/rangefeedcache/watcher.go:316 ⋮ [n1] 43 spanconfig-subscriber: established range feed cache
76 I260129 21:35:24.609901 86 1@util/log/event_log.go:32 ⋮ [n1] 44 ={"Timestamp":1769722524609885530,"EventType":"node_restart","NodeID":1,"StartedAt":1769722524505201799,"LastUp":1769715479113391252}
77 I260129 21:35:24.610060 86 sql/sqlliveness/slinstance/slinstance.go:313 ⋮ [n1] 45 starting SQL liveness instance
78 I260129 21:35:24.617193 914 sql/sqlliveness/slstorage/slstorage.go:460 ⋮ [n1] 46 inserted sqlliveness session 6c1c1bed5e8b466aad1c6346f4b08c1f
79 I260129 21:35:24.617290 914 sql/sqlliveness/slinstance/slinstance.go:198 ⋮ [n1] 47 created new SQL liveness session 6c1c1bed5e8b466aad1c6346f4b08c1f
80 I260129 21:35:24.617322 592 sql/sqlstats/persistedsqlstats/provider.go:132 ⋮ [n1] 48 starting sql-stats-worker with initial delay: 8m31.831156753s
81 I260129 21:35:24.617400 590 sql/temporary_schema.go:514 ⋮ [n1] 49 running temporary object cleanup background job
82 I260129 21:35:24.618578 947 kv/kvserver/replica_rangefeed.go:700 ⋮ [n1,rangefeed=‹lease›,s1,r6] 50 RangeFeed closed timestamp 1769715474.062948135,0 is behind by 1h57m30.55562578s
83 I260129 21:35:24.618974 112 kv/kvclient/rangefeed/rangefeedcache/watcher.go:316 ⋮ [n1] 51 settings-watcher: established range feed cache
84 I260129 21:35:24.619735 590 sql/temporary_schema.go:559 ⋮ [n1] 52 found 0 temporary schemas
85 I260129 21:35:24.619795 590 sql/temporary_schema.go:562 ⋮ [n1] 53 early exiting temporary schema cleaner as no temporary schemas were found
86 I260129 21:35:24.619817 590 sql/temporary_schema.go:563 ⋮ [n1] 54 completed temporary object cleanup job
87 I260129 21:35:24.619837 590 sql/temporary_schema.go:646 ⋮ [n1] 55 temporary object cleaner next scheduled to run at 2026-01-29 22:05:24.61722734 +0000 UTC
88 I260129 21:35:24.619742 86 server/server_sql.go:1295 ⋮ [n1] 56 done ensuring all necessary startup migrations have run
89 I260129 21:35:24.619938 963 kv/kvclient/rangefeed/rangefeedcache/watcher.go:316 ⋮ [n1] 57 system-config-cache: established range feed cache
90 I260129 21:35:24.620179 860 jobs/job_scheduler.go:433 ⋮ [n1] 58 waiting 4m0s before scheduled jobs daemon start
91 I260129 21:35:24.620876 896 kv/kvclient/rangefeed/rangefeedcache/watcher.go:316 ⋮ [n1] 59 tenant-settings-watcher: established range feed cache
92 I260129 21:35:24.629834 1043 sql/sqlliveness/slstorage/slstorage.go:342 ⋮ [n1,intExec=‹expire-sessions›] 60 deleted session c333e66ca5cd45379691a7a66d0ef687 which expired at 1769715517.453283391,0
93 I260129 21:35:24.635736 863 server/auto_upgrade.go:79 ⋮ [n1] 61 no need to upgrade, cluster already at the newest version
94 I260129 21:35:24.745459 86 1@server/server_sql.go:1365 ⋮ [n1] 62 serving sql connections
95 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 node startup completed:
96 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +CockroachDB node starting at 2026-01-29 21:35:24.745525439 +0000 UTC (took 1.2s)
97 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +build: OSS v22.1.22-46-g367bca413b @ 2025/10/08 03:53:50 (go1.22.11)
98 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +webui: ‹http://[::1]:53706›
99 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +sql: ‹postgresql://root@[::1]:40349/defaultdb?sslmode=disable›
100 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +sql (JDBC): ‹jdbc:postgresql://[::1]:40349/defaultdb?sslmode=disable&user=root›
101 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +RPC client flags: ‹cockroach <client cmd> --host=[::1]:40349 --insecure›
102 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +logs: ‹/var/tmp/omicron_tmp/.tmpzonC67/data/logs›
103 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +temp dir: ‹/var/tmp/omicron_tmp/.tmpzonC67/data/cockroach-temp1337910560›
104 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +external I/O path: ‹/var/tmp/omicron_tmp/.tmpzonC67/data/extern›
105 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +store[0]: ‹path=/var/tmp/omicron_tmp/.tmpzonC67/data›
106 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +storage engine: pebble
107 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +clusterID: ‹7ccbaf7e-e8bb-4a15-9a31-b9fcf76898ed›
108 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +status: restarted pre-existing node
109 I260129 21:35:24.745691 86 1@cli/start.go:1033 ⋮ [n1] 63 +nodeID: 1
110 I260129 21:35:24.813149 1069 util/log/event_log.go:32 ⋮ [n1,client=[::1]:43187,user=root] 64 ={"Timestamp":1769722524802148037,"EventType":"set_cluster_setting","Statement":"SET CLUSTER SETTING \"kv.raft_log.disable_synchronization_unsafe\" = true","Tag":"SET CLUSTER SETTING","User":"root","SettingName":"kv.raft_log.disable_synchronization_unsafe","Value":"‹true›"}
111 I260129 21:35:26.358698 1109 server/diagnostics/update_checker.go:150 ⋮ [n1] 65 a new version is available: ‹26.1.0-rc.1›, details: ‹https://www.cockroachlabs.com/docs/releases/v26.1#v26-1-0-rc-1›
112 I260129 21:35:26.395679 202 1@gossip/gossip.go:1547 ⋮ [n1] 66 node has connected to cluster via gossip
113 I260129 21:35:26.396313 202 kv/kvserver/stores.go:269 ⋮ [n1] 67 wrote 0 node addresses to persistent storage
114 E260129 21:35:27.945545 1110 server/status/recorder.go:420 ⋮ [n1] 68 could not get total system memory: ‹not implemented on illumos›
115 E260129 21:35:34.535391 140 1@server/status/runtime.go:454 ⋮ [n1] 69 unable to get mem usage: ‹not implemented on illumos›
116 E260129 21:35:34.535544 140 1@server/status/runtime.go:458 ⋮ [n1] 70 unable to get cpu usage: ‹not implemented on illumos›
117 W260129 21:35:34.535670 140 1@server/status/runtime.go:468 ⋮ [n1] 71 unable to get file descriptor usage (will not try again): ‹not implemented on illumos›
118 W260129 21:35:34.535723 140 1@server/status/runtime.go:478 ⋮ [n1] 72 problem fetching disk stats: ‹not implemented yet›; disk stats will be empty.
119 W260129 21:35:34.535742 140 1@server/status/runtime.go:499 ⋮ [n1] 73 problem fetching net stats: ‹not implemented yet›; net stats will be empty.
120 E260129 21:35:34.604126 284 server/status/recorder.go:420 ⋮ [n1,summaries] 74 could not get total system memory: ‹not implemented on illumos›
121 I260129 21:35:36.472997 1 1@cli/start.go:755 ⋮ [n1] 75 received signal 'terminated'
122 I260129 21:35:36.517830 1 1@cli/start.go:852 ⋮ [n1] 76 initiating graceful shutdown of server
123 I260129 21:35:36.517989 2943 1@server/drain.go:342 ⋮ [n1,server drain process] 77 waiting for health probes to notice that the node is not ready for new sql connections
124 I260129 21:35:36.518359 2943 sql/sqlstats/persistedsqlstats/flush.go:68 ⋮ [n1,server drain process] 78 flushing 192 stmt/txn fingerprints (215040 bytes) after 2.265µs
125 I260129 21:35:37.599234 2943 1@server/drain.go:285 ⋮ [n1,server drain process] 79 drain remaining: 173
126 I260129 21:35:37.599343 2943 1@server/drain.go:287 ⋮ [n1,server drain process] 80 drain details: descriptor leases: 172, liveness record: 1
127 I260129 21:35:37.801252 2943 sql/sqlstats/persistedsqlstats/flush.go:68 ⋮ [n1,server drain process] 81 flushing 2 stmt/txn fingerprints (10240 bytes) after 3.217µs
128 W260129 21:35:37.804683 2943 sql/sqlstats/persistedsqlstats/flush.go:170 ⋮ [n1,server drain process] 82 ‹failed to flush statement statistics›: flush statement ‹4025915790256006928›'s statistics: ‹insert-stmt-stats›: cannot acquire lease when draining
129 W260129 21:35:37.806065 2943 sql/sqlstats/persistedsqlstats/flush.go:170 ⋮ [n1,server drain process] 83 ‹failed to flush transaction statistics›: flushing transaction ‹11006041404254834895›'s statistics: ‹insert-txn-stats›: cannot acquire lease when draining
130 I260129 21:35:37.810628 2943 1@server/drain.go:285 ⋮ [n1,server drain process] 84 drain remaining: 0
131 I260129 21:35:37.811394 932 sql/stats/automatic_stats.go:489 ⋮ [n1] 85 quiescing auto stats refresher
132 W260129 21:35:37.812160 937 jobs/registry.go:815 ⋮ [n1] 86 canceling all adopted jobs due to stopper quiescing
133 W260129 21:35:37.812678 914 sql/sqlliveness/slinstance/slinstance.go:235 ⋮ [n1] 87 exiting heartbeat loop
134 I260129 21:35:37.818140 1 1@cli/start.go:904 ⋮ [n1] 88 server drained and shutdown completed