ColdFusion and Pagination

One of the more common tasks a web developer gets asked to do is add pagination to a result set. By pagination I simply mean displaying one "page" of content at a time. So if you had 22 records and wanted to show 10 at a time, there would be three pages of content. Let's take a look at one way to solve this problem.

First, let's get some data: <cfset data = queryNew("id,name,age,active","integer,varchar,integer,bit")>

<cfloop index="x" from="1" to="22">
   <cfset queryAddRow(data)>
   <cfset querySetCell(data,"id",x)>
   <cfset querySetCell(data,"name","User #x#")>
   <cfset querySetCell(data,"age",randRange(20,90))>
   <cfset querySetCell(data,"active",false)>
</cfloop>

I use queryNew to create a "fake" query. I then populate it with random data. Normally you would have a real cfquery here, but I think you get the point. Now I'm going to need to know how many items to show per page. I could hard code a number and use that, but I know it's best to abstract this into a variable:

<cfset perpage = 10>

I'd probably suggest an application variable actually as if you use pagination in one place, you will probably use it in multiple places, and you want to be consistent. Next I'm going to create a variable that will tell me what record I'll be starting with. This isn't the current page per se, but ends up being the same thing. So if we had 22 records, the first page will start with record 1. The second page will start with record 11. Here is the variable I will use along with the validation:

<cfparam name="url.start" default="1">
<cfif not isNumeric(url.start) or url.start lt 1 or url.start gt data.recordCount or round(url.start) neq url.start>
   <cfset url.start = 1>
</cfif>

There is a lot going on in that conditional, so let me break it out. IsNumeric will ensure that the URL variable is a number and not some other string like "apple." The lt 1 and gt data.recordCount simply ensures we are starting between 1 and the total number of rows in the query. Lastly, the round check simply ensures we have an integer value and not something like 10.2. Probably a bit overkill, but you can't be too careful with URL (and other client controlled) data.

Now let's display the data:

<h2>Random People</h2>

<cfoutput query="data" startrow="#url.start#" maxrows="#perpage#">
#currentrow#) #name#<br />
</cfoutput>

Nothing magic here. I simply tell cfoutput to loop from the starting index and stop after perpage records. Notice my lovely use of HTML. Ok, my design sucks, but you get the idea. Now let's do the pagination code:

<p align="right">
[
<cfif url.start gt 1>
   <cfset link = cgi.script_name & "?start=" & (url.start - perpage)>
   <cfoutput><a href="#link#">Previous Page</a></cfoutput>
<cfelse>
   Previous Page
</cfif>
/
<cfif (url.start + perpage - 1) lt data.recordCount>
   <cfset link = cgi.script_name & "?start=" & (url.start + perpage)>
   <cfoutput><a href="#link#">Next Page</a></cfoutput>
<cfelse>
   Next Page
</cfif>
]
</p>

Basically there are two things going on here, ignoring my simple HTML. The first cfif block checks to see if we need to make a linked or plain text "Previous Page" output. If url.start is above 1, then we need to make the previous link hot. I do this by checking the current script_name. I could have hard coded it as well. I then simply set start to the current value minus the number of entries per page. Note - a user could change url.start so that it is a low number, like 3. Then the value in the link would be negative. However, I already took care of that so I'm covered. This link doesn't handle other URL variables in the link. I'll cover that in a later blog entry if folks are interested.

The next cfif block is virtually the same as the first one. The logic here is to check if the current starting row, plus the per page value, minus one, is less then the total. Seems a bit complex, but the basic idea is to see if we have complete additional page of records to display.

That's it! Very quick and simple pagination.

Note: I edited the entry due to a bug found by Fernando. Thanks Fernando!

Comments

Great!

I've been using the pagination that was supplied in the CF WACK book, but this is another way for me to learn what's going on.
# Posted By Michael Walker II | 4/24/06 12:33 PM
<blockquote>I use queryNew to create a "fake" query.</blockquote>
I prefer to call it a "manual" query or more appropriately a "manual query result set". Bah. Semantics.
# Posted By djuggler | 4/24/06 1:31 PM
Nah, you are right. I'll start using that term as well.
# Posted By Raymond Camden | 4/24/06 1:37 PM
I like to do a variation upon this theme:

<cfparam name="startRow" default="1">
<cfparam name="rowsPerPage" default="20">

... cfquery or cfstoredproc ...

<cfset totalRecords = query.recordcount>

<cfset endRow = (startRow + rowsPerPage) - 1>

<!--- If the endrow is greater than the total, set the end row to to total --->
<cfif endRow GT totalRecords>
   <cfset endRow = totalRecords>
</cfif>

<!--- Add an extra page if you have leftovers --->
<cfif (totalRecords MOD rowsPerPage) GT 0>
   <cfset totalPages = totalPages + 1>
</cfif>

<!--- Display all of the pages --->
<cfloop from="1" to="#totalPages#" index="i">
      
   <cfset startRow = ((i - 1) * rowsPerPage) + 1>

   <a href="foo.cfm?startRow=#startRow#">#i#</a>      
   
</cfloop>

A little verbose, but it manages extra pages just fine. You can refine the process to calculate the boundaries, but the practice is similar.
# Posted By Teddy Payne | 4/24/06 1:56 PM
A little trick for your end row logic. Don't forget the max/min functions:

<cfset endRow = max(startRow+rowsperpage-1, totalrecords)>
# Posted By Raymond Camden | 4/24/06 2:01 PM
Oops, I meant min(...) :)
# Posted By Raymond Camden | 4/24/06 2:01 PM
That is exactly the refinement I was referring to. =)

Good stuff, RC.
# Posted By Teddy Payne | 4/24/06 2:46 PM
The problem is: it breaks when the result set has 21 records.
# Posted By Fernando da Silva Trevisan | 4/24/06 6:11 PM
(that is the problem with those #@#*@# "paginations" ever.
There's a lot of ways to solve it, like use ceiling(recordcount/perpage) to know the exact number of pages you will have - just to say, as I don't like to be the guy who points the problem and don't solve it ;)
# Posted By Fernando da Silva Trevisan | 4/24/06 6:13 PM
Dang it. I had meant to test an 'edge' condition like that and forgot. I'm very sorry to all my readers! I will try to fix this asap.
# Posted By Raymond Camden | 4/24/06 6:25 PM
Ok, I kinda remember this issue. This change to the cfif will fix it:

<cfif (url.start + perpage - 1) lt data.recordCount>

I'm going to update the blog entry in an hour or so. (Since printing doesn't show comments.)
# Posted By Raymond Camden | 4/24/06 6:28 PM
I try to keep pagination in the SQL level, never in CF. It is extremely inefficient to transfer 50k records each time I want to display the next block of 20... SQL is much more efficient in fetching only the block I need and transfer only the minimum required information back to the application server.

The advantage of doing it within CF is that it is generalized for all datasources, but IMO the consequences are not worth it. I use a generic stored proc (mssql) to paginate any of my tables or views. It's been working like a charm for years.

The code works perfectly for small datasets, but if you have anything in the thousands (or millions like me) ... not such a good idea :)
# Posted By Rob Gonda | 4/24/06 7:15 PM
Fernando, thanks for the bug report. I edited the entry.
# Posted By Raymond Camden | 4/24/06 8:21 PM
You're welcome, Ray. Your work for the community is memorable, *I* thank you!
Best wishes!
# Posted By Fernando da Silva Trevisan | 4/24/06 9:45 PM
Rob, care to share your generic MSSQL implementation with any of us? I'm very curious as to how you chose to implement it. We've played around with doing that here at our company, but never got it quite right.
# Posted By Jacob | 4/25/06 7:40 AM
I am curious as well about the MSSQL solution. I ahve seen pagination with dynamic SQL statements, numbered aggregated tables that are searched often and data hierarchies that resemble Celko's data hierarchy.
# Posted By Teddy Payne | 4/25/06 9:49 AM
If your using MySQL or PostgreSQL you can use the LIMIT and OFFSET in your SQL I have an example here: http://www.petefreitag.com/item/451.cfm
# Posted By Pete Freitag | 4/25/06 12:53 PM
For the equivalent of LIMIT and OFFSET with MS SQL Server, see this article:

http://msdn.microsoft.com/library/default.asp?url=...

It includes various methods for doing pagination in MSSQL Server:

- Select Top (usually not practical)
- User-Specific Records (SQL Query or Stored Procedure)
- Application-Wide Records (Stored Procedure)
# Posted By Tidy Technologies (Adam Fairbanks) | 4/25/06 4:46 PM
just posted my solution. Feel free to check it out:
http://www.robgonda.com/blog/index.cfm/2006/4/25/M...
# Posted By Rob Gonda | 4/25/06 10:56 PM
hi
i need ur help. I want to implement pagination using flash forms...
Can u please guide me in this matter
# Posted By Deepika | 5/2/06 4:40 AM
pagination is swell. there ar a tons of ways to do it, and it kind of depands on how fresh you need your data (caching or not, etc..)

I have a DELIMA!

How can I EFFICIENTLY implement a pagination system with the query has a 1 to many JOIN and the output is using GROUP?

The record count and start rows are NOT accurate to the groups of data being worked with....

for example, I could have a query that returns 5 rows, but in reality, it is 2 main rows, one with 2 JOINED rows and the other with 3 joined rows... pagination goes to hell real quickly!

Anyone go t a suggestion?
# Posted By imstillatwork | 8/14/06 12:18 AM
How to you add the number of the pages and on which page are you?

For example the initial page would be "Page 1 from 10", when you hit Next you would have "Page 2 from 10"...and so on...

Thanks.
# Posted By Mircea | 9/25/06 6:41 PM
You know what row you are starting on, it's the url variable..

you know how many rows per page...it's a variable in the script...

so if perpage = 10, and start = 21 (page 3) is:

Records #perpage# to #perpage+url.start#
# Posted By imstillatwork | 9/25/06 7:22 PM
Thanks, Ray
# Posted By Ryan LeTulle | 1/21/08 10:12 AM
How would I add a button to this schema to show "All" records on one page?
# Posted By Gene | 1/30/08 12:35 PM
The form would simply pass a value that you would check for, and when you display the query, you wouldn't filter by a page.
# Posted By Raymond Camden | 1/30/08 3:18 PM
I have a similar problem as 'imstillatwork'. I have a query that returns data that has one-to-many relationships. To output correctly, I use group="[variable]" to keep rows from outputting twice.

However, the RecordCount counts those double records, before I group and output. So, even though my page says

Records 1 - 50 of 250 records, when it really should say something like 1-50 of 185 records. The difference of 65 records is directly related to the fact that some records are foreign keys in a different table, and may refer to the 'main' record more than once. Is there an efficient way to count the rows after the query and the grouping?
# Posted By CODY RUSH | 7/23/08 3:39 PM
Ok, so I have a work around for my question above.

After my [query of a] query, I am simply looping through a <CFOUTPUT> grouping and setting a variable of how many actual rows (after being grouped) are to be displayed.

I know this can't be the most efficient way to do this, but it works for now. Any suggestions/improvements would be appreciated.


<cfset groupedRows = 0>
<cfoutput query="FilterDocs" group="fulldoc_id">
<cfset groupedRows = groupedRows+1>
</cfoutput>


Raymond, thanks for your website and your CF work.

Cody R.
# Posted By CODY RUSH | 7/24/08 3:01 PM
I'm in vacation mode now so my brain isn't all hear, but if I grok you right, I think keeping a simple counter is a fine way to do it.
# Posted By Raymond Camden | 7/24/08 8:22 PM
thanks for this, Ray. It just helped me.
# Posted By walt | 8/25/08 1:53 PM