1
0
mirror of https://github.com/django/django.git synced 2025-01-20 15:19:20 +00:00

131 Commits

Author SHA1 Message Date
Tim Graham
6a85c888bf Added supports_select_union skips in queries and aggregation tests. 2024-08-26 12:53:08 -03:00
Simon Charette
a16f13a866
Fixed #35643 -- Fixed a crash when ordering a QuerySet by a reference containing "__".
Regression in b0ad41198b3e333f57351e3fce5a1fb47f23f376.

Refs #34013. The initial logic did not consider that annotation aliases
can include lookup or transform separators.

Thanks Gert Van Gool for the report and Mariusz Felisiak for the review.
2024-08-02 16:21:12 -03:00
Chris Muthig
42b567ab4c Refs #35339 -- Updated Aggregate class to return consistent source expressions.
Refactored the filter and order_by expressions in the Aggregate class to
return a list of Expression (or None) values, ensuring that the list
item is always available and represents the filter expression.
For the PostgreSQL OrderableAggMixin, the returned list will always
include the filter and the order_by value as the last two elements.

Lastly, emtpy Q objects passed directly into aggregate objects using
Aggregate.filter in admin facets are filtered out when resolving the
expression to avoid errors in get_refs().

Thanks Simon Charette for the review.
2024-04-25 17:40:03 -03:00
Simon Charette
77278929c8 Fixed #35042 -- Fixed a count() crash on combined queries.
Regression in 59bea9efd2768102fc9d3aedda469502c218e9b7.

Thanks Marcin for the report.
2023-12-16 20:19:24 +01:00
Simon Charette
eea4f92f9a
Refs #34013 -- Registered instance lookups as documented in tests. 2023-12-16 20:05:36 +01:00
Simon Charette
b0ad41198b Fixed #34013 -- Added QuerySet.order_by() support for annotation transforms.
Thanks Eugene Morozov and Ben Nace for the reports.
2023-12-12 05:51:33 +01:00
Simon Charette
15cb3c262a Refs #34975 -- Complemented rhs filtering aggregations for __in lookup.
While this isn't a regression it's clear that similar logic should be
applied when dealing with lists of expressions passed as a lookup value.
2023-11-18 15:40:52 +01:00
Simon Charette
7530cf3900 Fixed #34975 -- Fixed crash of conditional aggregate() over aggregations.
Adjustments made to solve_lookup_type to defer the resolving of
references for summarized aggregates failed to account for similar
requirements for lookup values which can also reference annotations
through Aggregate.filter.

Regression in b181cae2e3697b2e53b5b67ac67e59f3b05a6f0d.

Refs #25307.

Thanks Sergey Nesterenko for the report.
2023-11-18 15:38:04 +01:00
David Sanders
b863c5ffde Fixed #34967 -- Fixed queryset crash when grouping by constants on SQLite < 3.39.
On SQLite < 3.39, this forces a GROUP BY clause with a HAVING clause
when no grouping is specified.

Co-authored-by: Simon Charette <charette.s@gmail.com>
2023-11-13 12:01:40 +01:00
Simon Charette
3b4a571275 Fixed #34798 -- Fixed QuerySet.aggregate() crash when referencing expressions containing subqueries.
Regression in 59bea9efd2768102fc9d3aedda469502c218e9b7,
complements e5c844d6f2a4ac6ae674d741b5f1fa2a688cedf4.

Refs #28477, #34551.

Thanks Haldun Komsuoglu for the report.
2023-10-16 05:37:30 +02:00
Mariusz Felisiak
c9b9a52edc
Fixed #34750 -- Fixed QuerySet.count() when grouping by unused multi-valued annotations.
Thanks Toan Vuong for the report.
Thanks Simon Charette for the review.

Regression in 59bea9efd2768102fc9d3aedda469502c218e9b7.
2023-08-01 16:16:28 +02:00
Simon Charette
4087367ba8 Fixed #34748 -- Fixed queryset crash when grouping by a reference in a subquery.
Regression in dd68af62b2b27ece50d434f6a351877212e15c3f.

Thanks Toan Vuong for the report.
2023-07-29 16:08:20 +02:00
Simon Charette
68912e4f6f Fixed #34717 -- Fixed QuerySet.aggregate() crash when referencing window functions.
Regression in 59bea9efd2768102fc9d3aedda469502c218e9b7.

Refs #28477.

Thanks younes-chaoui for the report.
2023-07-19 08:21:33 +02:00
Simon Charette
e5c844d6f2 Fixed #34551 -- Fixed QuerySet.aggregate() crash when referencing subqueries.
Regression in 59bea9efd2768102fc9d3aedda469502c218e9b7.

Refs #28477.

Thanks Denis Roldán and Mariusz for the test.
2023-05-23 06:25:58 +02:00
Simon Charette
2ee01747c3 Refs #34551 -- Fixed QuerySet.aggregate() crash on precending aggregation reference.
Regression in 1297c0d0d76a708017fe196b61a0ab324df76954.

Refs #31679.
2023-05-23 06:25:27 +02:00
Simon Charette
9daf8b4109 Fixed #34464 -- Fixed queryset aggregation over group by reference.
Regression in 59bea9efd2768102fc9d3aedda469502c218e9b7.

Refs #28477.

Thanks Ian Cubitt for the report.
2023-04-07 06:57:32 +02:00
Mariusz Felisiak
0e2649fdf4 Fixed #34255 -- Made PostgreSQL backend use client-side parameters binding with psycopg version 3.
Thanks Guillaume Andreu Sabater for the report.

Co-authored-by: Florian Apolloner <apollo13@users.noreply.github.com>
2023-01-17 08:24:08 +01:00
Simon Charette
dd68af62b2 Fixed #34176 -- Fixed grouping by ambiguous aliases.
Regression in b7b28c7c189615543218e81319473888bc46d831.

Refs #31377.

Thanks Shai Berger for the report and reviews.

test_aggregation_subquery_annotation_values_collision() has been
updated as queries that are explicitly grouped by a subquery should
always be grouped by it and not its outer columns even if its alias
collides with referenced table columns. This was not possible to
accomplish at the time 10866a10 landed because we didn't have compiler
level handling of colliding aliases.
2023-01-09 10:52:51 +01:00
Mariusz Felisiak
3b24a3fa33
Removed unnecessary commas in tests. 2022-12-21 11:41:29 +01:00
Simon Charette
1297c0d0d7 Fixed #31679 -- Delayed annotating aggregations.
By avoiding to annotate aggregations meant to be possibly pushed to an
outer query until their references are resolved it is possible to
aggregate over a query with the same alias.

Even if #34176 is a convoluted case to support, this refactor seems
worth it given the reduction in complexity it brings with regards to
annotation removal when performing a subquery pushdown.
2022-11-23 17:46:07 +01:00
Simon Charette
10037130c1
Refs #28477 -- Fixed handling aliased annotations on aggregation.
Just like when using .annotate(), the .alias() method will generate the
necessary JOINs to resolve the alias even if not selected.

Since these JOINs could be multi-valued non-selected aggregates must be
considered to require subquery wrapping as a GROUP BY is required to
combine duplicated tuples from the base table.

Regression in 59bea9efd2768102fc9d3aedda469502c218e9b7.
2022-11-14 05:45:33 +01:00
Simon Charette
59bea9efd2 Fixed #28477 -- Stripped unused annotations on aggregation.
Also avoid an unnecessary pushdown when aggregating over a query that doesn't
have aggregate annotations.
2022-11-09 13:22:14 +01:00
Simon Charette
967f8750ab Refs #27849 -- Fixed filtered aggregates crash on filters that match everything. 2022-11-07 09:05:24 +01:00
Simon Charette
77cf70ea96 Refs #27849 -- Added test for filtered aggregates with empty conditions. 2022-11-07 08:57:40 +01:00
Gregor Gärtner
f0c06f8ab7 Refs #33990 -- Renamed TransactionTestCase.assertQuerysetEqual() to assertQuerySetEqual().
Co-Authored-By: Michael Howitz <mh@gocept.com>
2022-10-08 08:07:38 +02:00
Mariusz Felisiak
d795259ea9
Replaced assertQuerysetEqual() to assertSequenceEqual()/assertCountEqual() where appropriate.
Follow up to 3f7b3275627385f8f7531fca01cdda50d4ec6b6e.
2022-10-07 13:05:35 +02:00
Simon Charette
b7b28c7c18 Refs #31150 -- Enabled implicit GROUP BY aliases.
This ensures implicit grouping from aggregate function annotations
groups by uncollapsed selected aliases if supported.

The feature is disabled on Oracle because it doesn't support it.
2022-10-06 11:51:32 +02:00
Simon Charette
3d734c09ff Refs #33992 -- Refactored subquery grouping logic.
This required moving the combined queries slicing logic to the compiler
in order to allow Query.exists() to be called at expression resolving
time.

It allowed for Query.exists() to be called at Exists() initialization
time and thus ensured that get_group_by_cols() was operating on the
terminal representation of the query that only has a single column
selected.
2022-10-06 11:38:03 +02:00
Simon Charette
32536b1324 Fixed #33992 -- Fixed queryset crash when aggregating over a group containing Exists.
A more in-depth solution is likely to make sure that we always GROUP BY
selected annotations or revisit how we use Query.exists() in the Exists
expression but that requires extra work that isn't suitable for a
backport.

Regression in e5a92d400acb4ca6a8e1375d1ab8121f2c7220be.

Thanks Fernando Flores Villaça for the report.
2022-09-08 05:50:02 +02:00
Mariusz Felisiak
eb3699ea77
Fixed #33718 -- Dropped support for MySQL 5.7. 2022-07-08 13:30:12 +02:00
marcperrinoptel
4282fd468f Fixed #33655 -- Removed unnecessary constant from GROUP BY clause for QuerySet.exists(). 2022-04-26 06:19:18 +02:00
Mariusz Felisiak
93cae5cb2f Fixed CVE-2022-28346 -- Protected QuerySet.annotate(), aggregate(), and extra() against SQL injection in column aliases.
Thanks Splunk team: Preston Elder, Jacob Davis, Jacob Moore,
Matt Hanson, David Briggs, and a security researcher: Danylo Dmytriiev
(DDV_UA) for the report.
2022-04-11 08:59:33 +02:00
Luke Plant
40b8a6174f Fixed #33397 -- Corrected resolving output_field for DateField/DateTimeField/TimeField/DurationFields.
This includes refactoring of CombinedExpression._resolve_output_field()
so it no longer uses the behavior inherited from Expression of guessing
same output type if argument types match, and instead we explicitly
define the output type of all supported operations.

This also makes nonsensical operations involving dates
(e.g. date + date) raise a FieldError, and adds support for
automatically inferring output_field for cases such as:
* date - date
* date + duration
* date - duration
* time + duration
* time - time
2022-03-31 11:05:23 +02:00
Carlton Gibson
bb61f0186d Refs #32365 -- Removed internal uses of utils.timezone.utc alias.
Remaining test case ensures that uses of the alias are mapped
canonically by the migration writer.
2022-03-24 06:29:50 +01:00
Nick Pope
847f46e9bf
Removed redundant QuerySet.all() calls in docs and tests.
Most QuerySet methods are mapped onto the Manager and, in general,
it isn't necessary to call .all() on the manager.
2022-02-22 10:29:38 +01:00
Mariusz Felisiak
7119f40c98 Refs #33476 -- Refactored code to strictly match 88 characters line length. 2022-02-07 20:37:05 +01:00
django-bot
9c19aff7c7 Refs #33476 -- Reformatted code with Black. 2022-02-07 20:37:05 +01:00
Mariusz Felisiak
71e7c8e737
Fixed #33468 -- Fixed QuerySet.aggregate() after annotate() crash on aggregates with default.
Thanks Adam Johnson for the report.
2022-01-31 11:33:24 +01:00
Simon Charette
e5a92d400a Fixed #33282 -- Fixed a crash when OR'ing subquery and aggregation lookups.
As a QuerySet resolves to Query the outer column references grouping logic
should be defined on the latter and proxied from Subquery for the cases where
get_group_by_cols is called on unresolved expressions.

Thanks Antonio Terceiro for the report and initial patch.
2021-12-02 07:23:33 +01:00
Hannes Ljungberg
a934d377af Fixed #33262 -- Fixed crash of conditional aggregation on Exists(). 2021-11-04 06:31:32 +01:00
David Wobrock
ad36a198a1 Fixed #33141 -- Renamed Expression.empty_aggregate_value to empty_result_set_value. 2021-09-29 12:58:01 +02:00
David Wobrock
691486a5cf Fixed #33073 -- Fixed queryset crash with aggregation and empty/extra queryset annotation. 2021-09-01 20:59:16 +02:00
Tim Graham
022d29c934
Refs #10929 -- Allowed NowUTC SQL customization for third-party backends. 2021-08-24 08:28:03 +02:00
Nick Pope
501a8db465 Fixed #10929 -- Added default argument to aggregates.
Thanks to Simon Charette and Adam Johnson for the reviews.
2021-07-19 13:04:27 +02:00
Simon Charette
9f3cce172f Refs #26430 -- Re-introduced empty aggregation optimization.
The introduction of the Expression.empty_aggregate_value interface
allows the compilation stage to enable the EmptyResultSet optimization
if all the aggregates expressions implement it.

This also removes unnecessary RegrCount/Count.convert_value() methods.
Disabling the empty result set aggregation optimization when it wasn't
appropriate prevented None returned for a Count aggregation value.

Thanks Nick Pope for the review.
2021-07-02 07:25:42 +02:00
Simon Charette
f3112fde98 Fixed #26430 -- Fixed coalesced aggregation of empty result sets.
Disable the EmptyResultSet optimization when performing aggregation as
it might interfere with coalescence.
2021-07-02 06:40:54 +02:00
Simon Charette
277eea8fcc Fixed #32478 -- Included nested columns referenced by subqueries in GROUP BY on aggregations.
Regression in fb3f034f1c63160c0ff13c609acd01c18be12f80.

Refs #31094, #31150.

Thanks Igor Pejic for the report.
2021-02-24 09:01:36 +01:00
Hasan Ramezani
275dd4ebba
Fixed #32178 -- Allowed database backends to skip tests and mark expected failures.
Co-authored-by: Tim Graham <timograham@gmail.com>
2020-12-10 18:00:57 +01:00
Ian Foote
8b040e3cbb Fixed #25534, Fixed #31639 -- Added support for transform references in expressions.
Thanks Mariusz Felisiak and Simon Charette for reviews.
2020-11-27 20:42:04 +01:00
Hasan Ramezani
3f7b327562 Fixed #31235 -- Made assertQuerysetEqual() compare querysets directly.
This also replaces assertQuerysetEqual() to
assertSequenceEqual()/assertCountEqual() where appropriate.

Co-authored-by: Peter Inglesby <peter.inglesby@gmail.com>
Co-authored-by: Mariusz Felisiak <felisiak.mariusz@gmail.com>
2020-11-06 09:24:50 +01:00