Traversing Large Datasets with PHP 5.5 Generators and yield

This year was all about tools like Laravel, Composer, and Git — but for this last post I wanted to zoom in on a language-level feature. I’ve been exploring the generator and yield keyword that shipped with PHP 5.5, and I want to share how useful it is when working with large datasets.

The Problem: Large Arrays Blowing Up Memory

Say you have a CSV file with 500,000 rows and you need to process them. The naive approach is to load everything into an array:

<?php
function csvSatirlariniOku($dosya)
{
    $satirlar = [];
    $f = fopen($dosya, 'r');
    while (($satir = fgetcsv($f)) !== false) {
        $satirlar[] = $satir;
    }
    fclose($f);
    return $satirlar;
}

$tumSatirlar = csvSatirlariniOku('buyuk.csv');
foreach ($tumSatirlar as $satir) {
    // işle
}

This code loads all 500,000 rows into memory at once. Fine for small files. On a large file, you’ll hit the memory_limit wall.

I ran into this on a real project: a client’s data import system where daily CSV files were reaching 300–400k rows. The script would die halfway through with Fatal error: Allowed memory size exhausted. Bumping memory_limit in php.ini looked like a fix, but it just kicked the can down the road — the file sizes kept growing.

What Is a Generator?

A generator is a type of function that produces values one at a time, on demand, instead of computing them all at once. The difference from a regular function is that it uses yield instead of return. It yields a value, pauses, and resumes from where it left off on the next call.

This was added to the language in PHP 5.5.

The Same Example with yield

<?php
function csvSatirlariniOkuGenerator($dosya)
{
    $f = fopen($dosya, 'r');
    while (($satir = fgetcsv($f)) !== false) {
        yield $satir;
    }
    fclose($f);
}

foreach (csvSatirlariniOkuGenerator('buyuk.csv') as $satir) {
    // Only one row in memory at a time
}

It looks like a small change, but the runtime behavior is completely different. Each time yield executes, it hands the current $satir value out and pauses the function. When foreach asks for the next value, the function picks up right where it left off. Instead of 500,000 rows in memory, there is always exactly one.

Number Range Example

A simple number-sequence generator to illustrate the point:

<?php
function aralik($baslangic, $bitis)
{
    for ($i = $baslangic; $i <= $bitis; $i++) {
        yield $i;
    }
}

foreach (aralik(1, 1000000) as $sayi) {
    echo $sayi . "\n";
}

PHP’s built-in range(1, 1000000) creates an array of one million elements and loads them all into memory. The generator version holds only the current number at each step.

The Generator Object

Calling a generator function does not execute it immediately — it returns a Generator object. You can iterate over it with foreach, or control it manually:

<?php
function ureticim()
{
    yield 'birinci';
    yield 'ikinci';
    yield 'ucuncu';
}

$uretec = ureticim();
echo $uretec->current(); // birinci
$uretec->next();
echo $uretec->current(); // ikinci

current() returns the current value; next() advances to the next step. In practice, this level of manual control is rarely needed — foreach handles it in almost every case.

Key-Value Pairs with yield

You can yield key-value pairs just like an associative array:

<?php
function veriUret()
{
    yield 'ad'    => 'Ahmet';
    yield 'email' => 'ahmet@ornek.com';
    yield 'yas'   => 30;
}

foreach (veriUret() as $anahtar => $deger) {
    echo "{$anahtar}: {$deger}\n";
}

Watch Out: Generators Cannot Be Rewound

You can only traverse a generator once. Once the foreach loop finishes, trying to loop over the same generator object again will produce nothing — the values are not re-generated. If you need to process the same data twice, you have to call the function again to get a fresh generator object. This is a fundamental difference from regular arrays, and forgetting it can cause silent, incorrect behavior.

When Should You Use It?

Using generators everywhere is overkill. If the dataset is small and fits comfortably in memory, a regular array is simpler.

Generators earn their keep when:

• Processing large files line by line
• Fetching a large number of database records when you don’t need them all at once
• Working with an infinite or very large sequence (API pagination, streaming data)

I learned this feature because I had a concrete need: a daily CSV import in one project kept hitting the memory limit. Generators solved it. Now, whenever I’m about to write a loop over a large dataset, this option is always somewhere in the back of my mind.

As the last post of 2014, I wanted to pick a slightly different topic. Next year I’ll be back to covering the new version of Laravel.