Neptune OpenCypher query sped up by toFloat - Why?

Question

I have a complex query, that I tried to speed up. The issue can be replicated with this query, on ~6 million Users:

`MATCH (u:User) WHERE u.score >= 0.75 RETURN u.id`

Using dynamic explain, I can see a bunch of rows with `DFEPipelineJoin (DFX)` and something like this (abbreviated):

```
pattern=projeut ?u ?u_score1 (?u,,?u_score1)
constraints=[(?u_score1 >= "0.75"^^xsd:double)] 
patternEstimate=14947926
```

And the same lines with `constraints=[(?u_score1 >= "0.75"^^xsd:float)] `, `constraints=[(?u_score1 >= "1"^^xsd:byte)] `, `constraints=[(?u_score1 >= "1"^^xsd:short)]` and `int` and `long`.
But it's always a floating point number (or NULL).
So I tried `USING QUERY:assumeConsistentDataTypes "true"`. I believe, it didn't change anything in my original query, so I tried something else. It did, however, change the plan for this query. I went back to my old query and it does change something.

Original query with dynamic explain: 80ms, with `assumeConsistentDataTypes`: 21ms

But bc. it didn't work the first time, I tried a few things. I checked whether all `score` properties really are floats, as best as I could - AWS Neptune doesn't seem to support the `valueType()` function.
Neptune also doesn't have a `toDouble` function, but a `toFloat` one, so I changed it to `WHERE toFloat(u.score) >= 0.75`. And that really sped up my query.

For this simple test query, it goes down to 17ms. But for my real query, we go from ~2.5 seconds to ~1.5 seconds.
I've also tried using both and it is not as fast as `toFloat` alone (quick test: 2.7 use consistent, 2.3 both, 1.7 toFloat)

So my queston: What is going on here? And is this sound? Can I use this in production without fear? 
Precision isn't really a concern - this is just a cut-off. As long as we are above or right around 0.75, it's fine.

One difference I see:
Using `use assumeConsistentDataTypes` still gives me a DFEPipelineJoin, but this time only a single one with an `xsd:double` constraint.
But with `toFloat`, I see `DFECreateConstant`, for the conversion that occurs later, but then I also see `DFEApply functor=ge(?104, ?101)` and `DFEApply functor=trueToRowId(?103)` - so it looks to me, like it applies a predicate function using the constants instead of performing a join.

Here is a gist with the query plans: https://gist.github.com/cptwunderlich/22605b11fb81bfa0400375b5c0f91913

re:Post Agent · Answer

The performance improvement you're seeing when using `toFloat(u.score) >= 0.75` instead of `u.score >= 0.75` is related to how Neptune processes type conversions and optimizes query execution.

When you use `u.score >= 0.75` without any type hints, Neptune has to handle potential type inconsistencies in your data. Even though you believe all values are floats, Neptune doesn't know this for certain. The query planner generates multiple constraint checks for different data types (`xsd:double`, `xsd:float`, `xsd:byte`, `xsd:short`, etc.) as shown in your query plan with multiple `DFEPipelineJoin` operations. This type of plan requires more processing overhead.

By explicitly using `toFloat(u.score)`, you're forcing a specific type conversion, which allows Neptune to generate a more efficient query plan. Instead of multiple join operations for different potential data types, it can use a direct function application (`DFEApply functor=ge`) with constants. This eliminates the need for multiple type-specific constraint checks.

The `USING QUERY:assumeConsistentDataTypes "true"` directive helps by telling Neptune that properties with the same name have consistent data types, which reduces some of the type checking overhead. However, explicitly converting with `toFloat()` seems to provide an even more optimized execution path in your case.

This optimization is safe to use in production as long as:
1. You're confident that all `score` values are numeric and can be converted to float
2. The precision differences between float and double don't affect your use case (which you mentioned is fine)
3. You've tested with representative data volumes

The performance gain comes from eliminating type ambiguity, allowing Neptune to generate a more streamlined execution plan with fewer operations. This is particularly beneficial when working with large datasets like your 6 million users.  
**Sources**  
[openCypher query best practices - Neptune Analytics](https://docs.aws.amazon.com/neptune-analytics/latest/userguide/best-practices-content.html)  
[openCypher extensions in Amazon Neptune - Amazon Neptune](https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-opencypher-extensions.html)

Neptune OpenCypher query sped up by toFloat - Why?

Add your answer

Relevant content