Crawling site pages to form cache on D7 and using software

To form a cache on the site and cut preview images, sometimes the task is set to bypass all the pages of the site. The easiest way to do this is in two ways: with your own script or using third-party software

  $ DOCUMENT_ROOT = $ _SERVER ['DOCUMENT_ROOT'];
$ domainName = 'http://www.site.ru';
$ iblockIds = [2,];
$ pageNum = 20;
 
define ('NO_KEEP_STATISTIC', true);
define ('NOT_CHECK_PERMISSIONS', true);
define ('CHK_EVENT', true);
 
require ($ _ SERVER ['DOCUMENT_ROOT']. '/bitrix/modules/main/include/prolog_before.php');
 
@set_time_limit (0);
@ignore_user_abort (true);
 
if (\ Bitrix \ Main \ Loader :: includeModule ('iblock')) {
 
    $ urls = array ();
 
    foreach ($ iblockIds as $ iblockId) {
 
        $ sectionIterator = CIBlockSection :: GetList (
            array (),
            array (
                'IBLOCK_ID' => $ iblockId,
                'ACTIVE' => 'Y',
                'GLOBAL_ACTIVE' => 'Y',
            ),
            array (
                'ELEMENT_SUBSECTIONS' => 'Y',
                'CNT_ACTIVE' => 'Y',
            ),
            (
                'ID',
                'IBLOCK_ID',
                'SECTION_PAGE_URL'
            )
        );
 
        while ($ section = $ sectionIterator-> GetNext ()) {
            for ($ i = 1, $ count = ceil ($ section ['ELEMENT_CNT'] / $ pageNum); $ i <= $ count; $ i ++) {
                $ urls [] = sprintf ('% s? PAGEN_1 =% d', $ domainName. $ section ['SECTION_PAGE_URL'], $ i);
            }
        }
 
        $ elementIterator = CIBlockElement :: GetList (
            array (),
            array (
                'IBLOCK_ID' => $ iblockId,
                'ACTIVE' => 'Y',
            ),
            false,
            false,
            array (
                'ID',
                'IBLOCK_ID',
                'DETAIL_PAGE_URL'
            )
        );
 
        while ($ element = $ elementIterator-> GetNext ()) {
             $ urls [] = $ domainName. $ element ['DETAIL_PAGE_URL'];
         }
     }
 
    foreach ($ urls as $ url) {
        @file_get_contents ($ url);
        @file_put_contents ($ _ SERVER ['DOCUMENT_ROOT']. '/import.log', $ url. "\ n", FILE_APPEND);
    }
}  

An example of running a php script from the console:

  php -f /var/www/site.ru/import.php &  

Before I came across this script, and now I also use the program SiteAnalyser , which not only automatically crawls all pages, but also generates any useful information for thought

• Теги: 1C BitrixCacheinit.phpD7
  Previous Article Sending SMS to a user on Bitrix

  Next Article Sort wordpress comment list to get tree view in ES6

  Back to list