480 Commits

Author SHA1 Message Date
Andrew Kane
1dc6514b66 Updated comment [skip ci] 2024-03-15 12:38:14 -07:00
Andrew Kane
6c53f7ca02 Updated comment [skip ci] 2024-03-15 12:37:47 -07:00
Heikki Linnakangas
0d35a14198 Fix compiler warnings in strict C99 mode (#487)
Redefining a typedef is a C11 feature:

    In file included from src/hnsw.c:10:
    src/hnsw.h:147:5: warning: redefinition of typedef 'HnswElementData' is a C11 feature [-Wtypedef-redefinition]
    }                       HnswElementData;
                            ^
    src/hnsw.h:118:32: note: previous definition is here
    typedef struct HnswElementData HnswElementData;
                                   ^
    src/hnsw.h:163:5: warning: redefinition of typedef 'HnswNeighborArray' is a C11 feature [-Wtypedef-redefinition]
    }                       HnswNeighborArray;
                            ^
    src/hnsw.h:119:34: note: previous definition is here
    typedef struct HnswNeighborArray HnswNeighborArray;
                                     ^
    2 warnings generated.

I got these warnings when I built PostgreSQL with "CC=clang
CFLAGS=-std=gnu99"; other similar options would surely produce the
warnings too.
2024-03-12 02:02:33 -07:00
Andrew Kane
3ea2ce89be Reduced lock contention with parallel HNSW index builds 2024-03-11 20:16:55 -07:00
Andrew Kane
91e3d2905f Fixed sort function for Postgres 12 2024-02-28 16:26:41 -08:00
Andrew Kane
fe2406564f Replaced pairing heap with array in SelectNeighbors - closes #447
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi>
2024-02-28 15:47:26 -08:00
Andrew Kane
fa52511eaa Fixed closer caching for Postgres 12 2024-02-28 15:44:38 -08:00
Andrew Kane
b447ae4989 Avoid base address for relptr for Postgres < 14.5 2024-02-28 14:10:14 -08:00
Andrew Kane
efed873a3e Revert "Replaced pairing heap with array in SelectNeighbors - closes #447"
This reverts commit 14b278dec9.
2024-02-28 11:33:14 -08:00
Andrew Kane
14b278dec9 Replaced pairing heap with array in SelectNeighbors - closes #447
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi>
2024-02-28 11:25:12 -08:00
Andrew Kane
ca10cbaa7d Revert "Remove offsethash"
This reverts commit 1cbd204f52.
2024-02-20 16:07:32 -08:00
Andrew Kane
eb29019a14 Revert "Eliminate a few HnswPtrAccess invocations"
This reverts commit 334c386a45.
2024-02-20 16:07:21 -08:00
Heikki Linnakangas
334c386a45 Eliminate a few HnswPtrAccess invocations
HnswPtrAccess() is pretty cheap, but it stills seems worthwhile to
avoid repeated calls in the hot paths when it can be easily avoided.
2024-02-19 14:13:18 +02:00
Heikki Linnakangas
1cbd204f52 Remove offsethash
The original motivation was to eliminate the superfluous HnswPtrAccess
call from AddToVisited. The caller has to call HnswPtrAccess() anyway,
so it makes sense to pass the HnswElement rather than HnswElementPtr
to AddToVisited(). But then I realized that we can use the
pointer-variant even with shared memory, because the visited-hash is
backend-private, and the addresses where the elements are mapped to in
shared memory are stable within the backend.
2024-02-19 14:12:26 +02:00
Andrew Kane
5ba62fca84 Fixed crash with shared_preload_libraries - fixes #460 2024-02-14 17:13:30 -08:00
Andrew Kane
f43cd0ed98 Improved type [skip ci] 2024-02-05 19:38:52 -08:00
Andrew Kane
2716a223a6 Fixed error with ANALYZE and vectors with different dimensions - fixes #451 2024-02-02 10:47:48 -08:00
Andrew Kane
5b43aaad5a Removed unneeded item ids 2024-01-28 23:14:52 -08:00
Andrew Kane
797ce8034c Ran pgindent [skip ci] 2024-01-28 23:03:48 -08:00
Andrew Kane
cad9e22d9a Updated variable name to be consistent with CreateGraphPages [skip ci] 2024-01-28 23:02:13 -08:00
Andrew Kane
a7d43904f7 Zero memory for each element 2024-01-28 23:00:39 -08:00
Andrew Kane
ded649891b DRY HNSW tuple alloc size [skip ci] 2024-01-28 22:58:38 -08:00
Heikki Linnakangas
2d092016fc Remove unnecessary PageIndexTupleOverwrite calls that caused UB (#438)
These places called PageIndexTupleOverwrite(), with the new tuple
pointing directly to the original page. The PageIndexTupleOverwrite()
call is unnecessary in these cases, as we have already modified the
tuple on the page directly. Moreover, PageIndexTupleOverWrite() will
call memcpy with same src and dst arguments, which is undefined
behavior.

It's OK to modify the pages on disk directly, and we don't need
critical sections, because we either use the generic xlog functions
which create a temporary copy of the page, or we are building a new
index so if we crash the whole index is invisible and will be dropped
anyway.
2024-01-28 22:52:56 -08:00
Andrew Kane
86b31fdf96 Revert "Update neighbor tuples in-place for HNSW index build"
This reverts commit 270dd8189a.
2024-01-28 22:18:22 -08:00
Andrew Kane
642ee1f423 Improved function names 2024-01-28 21:00:12 -08:00
Andrew Kane
270dd8189a Update neighbor tuples in-place for HNSW index build 2024-01-28 20:53:56 -08:00
Andrew Kane
ba2776850b Fixed Valgrind check for HNSW in-memory, parallel index builds - closes #441 2024-01-28 16:19:32 -08:00
Andrew Kane
0cc883b944 Removed checking neighbors for cached distance 2024-01-28 02:11:24 -08:00
Andrew Kane
10416b841f Fixed uninitialized memory in scan->xs_recheck [skip ci] 2024-01-27 19:54:16 -08:00
Andrew Kane
d0f13d4e7f Fixed possibly uninitialized memory in lists [skip ci] 2024-01-27 19:51:34 -08:00
Andrew Kane
705e71015f Fixed uninitialized entry point level - fixes #439 and closes #440 2024-01-27 19:49:40 -08:00
Heikki Linnakangas
121f53b8ff Remove unused heapRel arguments (#443) 2024-01-26 12:45:10 -08:00
Heikki Linnakangas
571697fee7 Remove unnecessary UpdateProgress() wrappers (#433)
Now that we require PostgreSQL v12, we can use
pgstat_progress_update_param directly.
2024-01-25 00:07:57 -08:00
Andrew Kane
f7eda7bd20 DRY neighbor array size [skip ci] 2024-01-24 17:54:33 -08:00
Andrew Kane
2260e13315 Condensed code [skip ci] 2024-01-24 16:20:00 -08:00
Andrew Kane
90e0a14bda Moved allocating neighbor array to separate function [skip ci] 2024-01-24 16:17:34 -08:00
Heikki Linnakangas
c8be3a369b Include generic_xlog.h directly in the .c files where it's needed
There are no references to anything that's in generic_xlog.h in the
header files.
2024-01-23 13:04:03 +02:00
Heikki Linnakangas
e5d1a6bdbb Include reloptions.h directly in the .c files where it's needed
There are no references to anything that's in reloptions.h in the
header files. They need to include genam.h instead, which defines
IndexScanDesc.
2024-01-23 13:02:24 +02:00
Heikki Linnakangas
f31d708c2b Add direct include to pairingheap.h in headers
ivfflat.h and hnsw.h have references to pairingheap_node, so they need
to include lib/pairingheap.h. It happened to work, because
lib/pairingheap.h was being included indirectly through
nodes/execnodes.h, but let's be explicit.

Remove the include from hnswbuild.c, because there are no calls to
pairingheap functions in that file. Instead, add the includes to
hnswutils.c and ivfscan.c, which do have such calls. They are not
strictly necessary again because of the indirect include from hnsw.h
and ivfflat.h, but let's be explicit while we're messing with this.
2024-01-23 12:53:22 +02:00
Heikki Linnakangas
a1b1c99ff7 Remove unused #include
pg_list.h has not been used in hnswbuild.c since commit cb4c770df2.
2024-01-23 12:53:13 +02:00
Andrew Kane
083008c21e Added validation for GUC parameters 2024-01-22 23:55:30 -08:00
Andrew Kane
a1e526ef82 Dropped support for Postgres 11 2024-01-22 23:52:54 -08:00
Andrew Kane
8ffb3718a4 Leave more space for other shared memory 2024-01-22 23:31:55 -08:00
Andrew Kane
2d0f162bd7 Added support for in-memory parallel index builds for HNSW
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi>
2024-01-22 23:19:10 -08:00
Heikki Linnakangas
4c6928bd3c Remove HnswSpool
It was just used to pass heap/index relations to
HnswParallelScanAndInsert. I think it was copied from nbtsort.c, which
is more complicated. I don't think we need a struct like this.

(That said, I actually think that we should have a state object that
would hold fields like 'heap', 'index', 'procinfo', 'collation'
etc. Passing that object around would simplify the signatures of many
functions. But that's a different story).
2024-01-22 23:11:25 -08:00
Heikki Linnakangas
6fd05dd6f6 Remove unused 'scantuplesortstates' field 2024-01-22 23:08:20 -08:00
Andrew Kane
44b90be452 Made variable name consistent across functions [skip ci] 2024-01-22 19:02:33 -08:00
Andrew Kane
31572a7b28 Removed unused parameter [skip ci] 2024-01-22 19:00:45 -08:00
Andrew Kane
2427290ea9 Pass hash by reference 2024-01-22 18:34:40 -08:00
Andrew Kane
1ff9ab5133 Revert "Improved vector_in"
This reverts commit 4894dc5da1.
2024-01-17 17:14:42 -08:00