Beware of Array Spread
Let’s say you’re pulling ids from a database in batches and you want to combine these batches into one big array.
One approach might use spread with Array::push
:
const allUserIds: string[] = []
for await (const batchUserIds of getUserIds()) {
allUserIds.push(...batchUserIds)
}
// use allUserIds
But there’s a problem, if batchUserIds
is too long, JS will raise a RangeError
!
The max number of arguments, and in turn the max length of batchUserIds
, depends on the JS implementation.
- JavaScriptCore (Safari): max of 65,537 arguments
- SpiderMonkey (Firefox): max of 500,000 arguments
- V8 (Chrome, Node): depends on stack size
So there are somewhat abitrary limits that depend on the browser, which MDN warns about.
MDN is also kind enough to offer a solution, use smaller chunks when using .apply
or ...
(spread).
And if we update our code with a little help from Lodash’s chunk
:
const MAX_JS_ARGS = 65_537
const MAX_SPREAD_BATCH_SIZE = Math.floor(MAX_JS_ARGS / 2)
const allUserIds: string[] = []
for await (const batchUserIds of getUserIds()) {
chunk(batchUserIds, MAX_SPREAD_BATCH_SIZE).forEach(subBatch => {
allUserIds.push(...subBatch)
})
}
// use allUserIds
No more RangeError
!
TL;DR: watch out for large arrays when spreading