Searching for a Large Number of Objects






Searching for a Large Number of Objects

Problem

Your search is returning exactly 1,000 objects, which is only a subset of the objects you expected, and you want it to return all matching objects.

Solution

You might notice that searches with large numbers of matches stop displaying after 1,000. Domain controllers return only a maximum of 1,000 entries from a search unless paging is enabled. This is done to prevent queries from consuming a lot of resources on domain controllers by retrieving the results all at once instead of in pages or batches. The following examples are variations of Recipe 4.8, which will show how to enable paging and return all matching entries.

Using a graphical user interface
  1. Open LDP from the Windows Support Tools.

  2. From the menu, select Connection Connect.

  3. For Server, enter the name of a domain controller (or leave blank to do a serverless bind).

  4. For Port, enter 389.

  5. Click OK.

  6. From the menu, select Connection Bind.

  7. Enter the credentials of a user.

  8. Click OK.

  9. From the menu, select Browse Search.

  10. For BaseDN, type the base distinguished name of where the search will start. (You can leave this blank if you wish to connect to the domain NC as the base DN.)

  11. For Scope, select the appropriate scope.

  12. For Filter, enter an LDAP filter.

  13. Click Options to customize the options for this query.

  14. For Timeout (s), enter a value such as 10.

  15. For Page size, enter the number of objects to be returned with each page (e.g., 1,000).

  16. Under Search Call Type, select Paged.

  17. Click OK and then Run to perform the query. A page of results (i.e., 1,000 entries) will be displayed each time you click Run until all results have been returned.

Using a command-line interface
	> dsquery * <BaseDN> -limit 0 -scope <Scope> -filter "<Filter>" -attr "<AttrList>"

Using VBScript
	' This code enables  
paged searching
	' ------ SCRIPT CONFIGURATION ------
	strBase   = "<LDAP://<BaseDN>>;"
	strFilter = "<Filter>;"
	strAttrs  = "<AttrList>;"
	strScope  = "<Scope>"
	' ------ END CONFIGURATION ---------
	set objConn = CreateObject("ADODB.Connection")
	objConn.Provider = "ADsDSOObject"
	objConn.Open "Active Directory Provider"
	set objComm = CreateObject("ADODB.Command")
	objComm.ActiveConnection = objConn
	objComm.Properties("Page Size") = 1000
	objComm.CommandText = strBase & strFilter & strAttrs & strScope
	set objRS = objComm.Execute
	objRS.MoveFirst
	while Not objRS.EOF
	    Wscript.Echo objRS.Fields(0).Value
	    objRS.MoveNext
	wend

Discussion

Paged searching support is implemented via an LDAP control. LDAP controls were defined in RFC 2251 and the Paged control in RFC 2696. Controls are extensions to LDAP that were not built into the protocol, so not all directory vendors support the same ones.

In Active Directory, you can change the default maximum page size of 1,000 by modifying the LDAP query policy. See Recipe 4.27 for more information.


If you need searches to return hundreds of thousands of entries, Active Directory will return a maximum of only 262,144 entries even when paged searching is enabled. This value is defined in the LDAP query policy and can be modified like the maximum page size (see Recipe 4.27).

Using a graphical user interface

A word of caution when using LDAP to display a large number of entriesby default, only 2,048 lines will be displayed in the right pane. To change that value, go to Options General and change the Line Value under Buffer Size to a larger number.

Using a command-line interface

The only difference between this solution and Recipe 4.8 is the addition of the -limit 0 flag. With -limit set to 0, paging will be enabled according to the default LDAP query policy; matching objects will be returned within those parameters. If -limit is not specified, a maximum of 100 entries will be returned.

AdFind will return a large number of objects from a query without any modification.


Using VBScript

To enable paged searching in ADO, you must instantiate an ADO Command object. A Command object allows for various properties of a query to be set, such as size limit, time limit, and page size. See MSDN for the complete list.

See Also

Recipe 4.8 for searching for objects, Recipe 4.27 for viewing the default LDAP policy, RFC 2251 (Lightweight Directory Access Protocol (v3)), RFC 2696 (LDAP Control Extension for Simple Paged Results Manipulation), and MSDN: Searching with ActiveX Data Objects (ADO)



 Python   SQL   Java   php   Perl 
 game development   web development   internet   *nix   graphics   hardware 
 telecommunications   C++ 
 Flash   Active Directory   Windows