Andrew Kane
2d0f162bd7
Added support for in-memory parallel index builds for HNSW
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2024-01-22 23:19:10 -08:00
Heikki Linnakangas
4c6928bd3c
Remove HnswSpool
...
It was just used to pass heap/index relations to
HnswParallelScanAndInsert. I think it was copied from nbtsort.c, which
is more complicated. I don't think we need a struct like this.
(That said, I actually think that we should have a state object that
would hold fields like 'heap', 'index', 'procinfo', 'collation'
etc. Passing that object around would simplify the signatures of many
functions. But that's a different story).
2024-01-22 23:11:25 -08:00
Heikki Linnakangas
6fd05dd6f6
Remove unused 'scantuplesortstates' field
2024-01-22 23:08:20 -08:00
Andrew Kane
44b90be452
Made variable name consistent across functions [skip ci]
2024-01-22 19:02:33 -08:00
Andrew Kane
c7d60346d8
Improved macro [skip ci]
2024-01-13 20:02:41 -08:00
Andrew Kane
597bfdc76b
Added HnswGetNeighbors macro
2024-01-13 20:00:34 -08:00
Andrew Kane
cbf3eb4fa5
Improved HNSW build and insert code
2024-01-13 10:07:42 -08:00
Andrew Kane
1881b857f9
Simplified code
2024-01-09 18:53:31 -08:00
Andrew Kane
9e680884bd
Moved indtuples to HnswGraph
2024-01-07 22:23:49 -08:00
Andrew Kane
19a0e1b341
Moved graph to separate struct
2024-01-07 20:15:30 -08:00
Andrew Kane
cb4c770df2
Switched to slist for elements to reduce allocations and remove limit
2024-01-07 18:26:19 -08:00
Andrew Kane
8ee37b60a0
Improved memory estimate for HNSW index builds
2024-01-03 13:47:50 -05:00
Andrew Kane
9b73b3d1a6
Reduced memory and allocations for heap TIDs - closes #385
2024-01-03 13:41:34 -05:00
Andrew Kane
e8c3bf0cef
Improved memory tracking for HNSW index builds - #384
2023-12-22 13:35:43 -05:00
Andrew Kane
dcbe0b6f0d
Reduced memory usage for HNSW index builds - #384
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2023-12-22 12:41:47 -05:00
Andrew Kane
6738fa0bd7
Added HNSW_MEMORY flag to show memory usage - #384 [skip ci]
2023-12-20 16:49:16 -05:00
Andrew Kane
e88a425c9b
Reduced WAL generation for HNSW index builds - thanks @hlinnaka
2023-12-19 20:37:32 -05:00
Andrew Kane
921427ee03
Replace dynahash hash table in HNSW with simplehash for speed - #378
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2023-12-17 11:24:13 -05:00
Andrew Kane
dfee5d4045
Added support for on-disk parallel index builds for HNSW
2023-11-11 19:29:45 -08:00
Andrew Kane
94f7304ccd
Keep vector for now to be overly cautious about packing [skip ci]
2023-11-09 18:43:55 -08:00
Andrew Kane
d078db3d25
Switched HnswElementTuple to generic data and zero full section
2023-11-09 18:28:25 -08:00
Andrew Kane
3cf6f62900
Switched to datum for HnswElement
2023-11-09 17:35:39 -08:00
Andrew Kane
84e073888c
Removed vector-specific code from HNSW_ELEMENT_TUPLE_SIZE [skip ci]
2023-11-09 16:57:01 -08:00
Andrew Kane
3f3463bde5
Improved memory calculation for HNSW and removed vector-specific code
2023-11-09 16:21:26 -08:00
Andrew Kane
a713e2acaa
Improved performance of HNSW index builds - closes #292
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2023-10-05 13:21:26 -07:00
Andrew Kane
6e1312ddbe
DRY max size
2023-10-04 21:43:34 -07:00
Andrew Kane
bca50a03fa
Use consistent variable name
2023-09-12 19:24:31 -07:00
Andrew Kane
b164833933
Removed pinning for HNSW index scan
2023-09-11 12:12:28 -07:00
Andrew Kane
d05d6ee83d
Get m from metapage
2023-09-03 01:35:21 -07:00
Jonathan S. Katz
e50a79108f
Set default HNSW "ef_construction" to 64 ( #230 )
2023-08-26 16:05:08 -07:00
Andrew Kane
e406b7f5ea
Added comments [skip ci]
2023-08-23 21:03:07 -07:00
Andrew Kane
6d88a9e1d2
Updated HNSW_PAGE_ID [skip ci]
2023-08-21 22:59:53 -07:00
Andrew Kane
1e851c12c0
Updated comment [skip ci]
2023-08-21 22:55:24 -07:00
Andrew Kane
8ed3cc5f0b
Improved macro [skip ci]
2023-08-21 22:52:58 -07:00
Andrew Kane
bace0891bd
Updated comments [skip ci]
2023-08-21 22:51:24 -07:00
Andrew Kane
0e1de45463
Improved locking code [skip ci]
2023-08-21 15:42:59 -07:00
Andrew Kane
3f3b3ca8e3
Made function static [skip ci]
2023-08-21 03:27:42 -07:00
Andrew Kane
02f4e0ec8b
Revert "Added version to reduce stale reads and writes and prepare for optimistic locking"
...
This reverts commit ef1209eaf4
2023-08-21 02:47:27 -07:00
Andrew Kane
ef1209eaf4
Added version to reduce stale reads and writes and prepare for optimistic locking
2023-08-20 17:08:20 -07:00
Andrew Kane
687263ccd4
DRY HNSW vacuum code
2023-08-20 14:52:31 -07:00
Andrew Kane
a03f6ae4bc
Added prefix to function name [skip ci]
2023-08-18 00:54:09 -07:00
Andrew Kane
b72a22b3c0
Fixed duplicate connections when repairing graph
2023-08-16 17:07:19 -07:00
Andrew Kane
c3bafc76e8
Only update entry point on inserts if level is greater
2023-08-15 12:31:06 -07:00
Andrew Kane
e1d6654063
Revert "Improved HNSW vacuum performance"
...
This reverts commit c5b2f3ac8b
2023-08-13 17:19:22 -07:00
Andrew Kane
c5b2f3ac8b
Improved HNSW vacuum performance
2023-08-13 16:50:36 -07:00
Andrew Kane
0f238b1fa9
Update neighbors when vacuuming
2023-08-13 16:17:30 -07:00
Andrew Kane
c64288377b
Updated min ef_construction to 4 [skip ci]
2023-08-10 21:11:10 -07:00
Andrew Kane
3ff2e34d7f
Updated min M to 2 [skip ci]
2023-08-10 20:57:50 -07:00
Andrew Kane
1a0d7bccc7
Updated min ef_search to 1 [skip ci]
2023-08-10 20:47:15 -07:00
Andrew Kane
9b967d592f
Improved concurrent inserts with empty entry point, part 2
2023-08-10 18:31:58 -07:00
