[kou]

GitHub: Fix GH-378

Since CSV#shift re-enters CSV::Parser#parse for each row, parse_quotable_loose left one keep per row in the scanner. Each keep retains a StringScanner and its chunk.

A keep was also leaked when an exception was raised while parsing a row. If the caller rescued it and continued reading from IO, every following keep_drop merged its buffer into the leaked keep.

Use ensure to always drop the keep unless it was consumed by keep_back for switching to parse_quotable_robust.

Reported by sswander. Thanks!!!

[kou]

@sswander Could you try this?

[sswander]

@kou thanks for the quick fix!

Seems to work now.

Repro script:

# repro.rb
require "bundler/inline"

gemfile do
  source "https://rubygems.org"
  gem "csv", github: "kou/csv", branch: "fix-shift-keeps-leak"
end

require "csv"
require "tmpdir"

path = File.join(Dir.tmpdir, "repro.csv")
File.write(path, "a,b\n" * 1_000_000)

GC.start
before = GC.stat(:heap_live_slots)

csv = CSV.new(File.open(path))
nil while csv.gets

GC.start(full_mark: true, immediate_sweep: true)
puts "csv #{CSV::VERSION}: retained after full GC = #{GC.stat(:heap_live_slots) - before}"

running ruby repro.rb results in

csv 3.3.6: retained after full GC = 163

[kou]

Thanks.
Merged and released.