Crawling site pages to form cache on D7 and using software

To form a cache on the site and cut preview images, sometimes the task is set to bypass all the pages of the site. The easiest way to do this is in two ways: with your own script or using third-party software

  $ DOCUMENT_ROOT = $ _SERVER ['DOCUMENT_ROOT'];
$ domainName = 'http://www.site.ru';
$ iblockIds = [2,];
$ pageNum = 20;
 
define ('NO_KEEP_STATISTIC', true);
define ('NOT_CHECK_PERMISSIONS', true);
define ('CHK_EVENT', true);
 
require ($ _ SERVER ['DOCUMENT_ROOT']. '/bitrix/modules/main/include/prolog_before.php');
 
@set_time_limit (0);
@ignore_user_abort (true);
 
if (\ Bitrix \ Main \ Loader :: includeModule ('iblock')) {
 
    $ urls = array ();
 
    foreach ($ iblockIds as $ iblockId) {
 
        $ sectionIterator = CIBlockSection :: GetList (
            array (),
            array (
                'IBLOCK_ID' => $ iblockId,
                'ACTIVE' => 'Y',
                'GLOBAL_ACTIVE' => 'Y',
            ),
            array (
                'ELEMENT_SUBSECTIONS' => 'Y',
                'CNT_ACTIVE' => 'Y',
            ),
            (
                'ID',
                'IBLOCK_ID',
                'SECTION_PAGE_URL'
            )
        );
 
        while ($ section = $ sectionIterator-> GetNext ()) {
            for ($ i = 1, $ count = ceil ($ section ['ELEMENT_CNT'] / $ pageNum); $ i <= $ count; $ i ++) {
                $ urls [] = sprintf ('% s? PAGEN_1 =% d', $ domainName. $ section ['SECTION_PAGE_URL'], $ i);
            }
        }
 
        $ elementIterator = CIBlockElement :: GetList (
            array (),
            array (
                'IBLOCK_ID' => $ iblockId,
                'ACTIVE' => 'Y',
            ),
            false,
            false,
            array (
                'ID',
                'IBLOCK_ID',
                'DETAIL_PAGE_URL'
            )
        );
 
        while ($ element = $ elementIterator-> GetNext ()) {
             $ urls [] = $ domainName. $ element ['DETAIL_PAGE_URL'];
         }
     }
 
    foreach ($ urls as $ url) {
        @file_get_contents ($ url);
        @file_put_contents ($ _ SERVER ['DOCUMENT_ROOT']. '/import.log', $ url. "\ n", FILE_APPEND);
    }
}  

An example of running a php script from the console:

  php -f /var/www/site.ru/import.php &  

Before I came across this script, and now I also use the program SiteAnalyser , which not only automatically crawls all pages, but also generates any useful information for thought

• Теги: 1C BitrixCacheinit.phpD7
  

  Author

  Aleksandr Tichiy

  Fullstack developer · 1C-Bitrix

  I build and modernize online stores and corporate portals on 1C-Bitrix. Integrations with 1C, high-load performance, 152-FZ compliance. Writing about architecture and e-commerce.

  Call Email Telegram Max