1. sphinx配置和使用

1.1. sphinx安装配置

1.1.1. 安装sphinx

sphinx下载官网:http://sphinxsearch.com/downloads/release/

sphinx如果要用mysql作为数据源的话需要依赖安装msyql-devel

wget http://sphinxsearch.com/files/sphinx-2.2.11-release.tar.gz
tar zxvf sphinx-2.2.11-release.tar.gz
cd sphinx-2.2.11-release
./configure --prefix=/usr/local/webserver/sphinx --with-mysql=/usr/local/webserver/mysql/
make && make install

注释:
--prefix : 指定Sphinx安装到何处,我的安装目录是“/usr/local/webserver/sphinx”
--with-mysql : mysql的安装目录,我的安装目录是“/usr/local/webserver/mysql/”
如果安装在系统默认则不需要指定

如果有以下报错:

../config/depcomp: line 669: exec: g++: not found
make[2]: *** [sphinx.o] 错误 127
make[2]: Leaving directory `/home/wuwei/sphinx-2.2.11-release/src'
make[1]: *** [all] 错误 2
make[1]: Leaving directory `/home/wuwei/sphinx-2.2.11-release/src'
make: *** [all-recursive] 错误 1

则提示g++: not found g++没有找到需要安装g++,在不同的linux中g++的名称可能不同

1.1.2. 配置sphinx

BBC提供的sphinx默认配置文件说明

BBC提供的sphinx默认配置目录:script/sphinx/

需要PHP支持的sphinx配置文件:script/sphinx/sphinx.conf

注意:sphinx服务器需要有PHP支持
该配置文件的好处:可以单独配置索引 索引配置存放在conf.d文件夹中

sphinx原生支持的配置文件:script/sphinx/sphinx_all.conf

 不需要PHP支持,所有索引配置在同一个文件中

选择配置文件script/sphinx/sphinx.conf + conf.d/*script/sphinx/sphinx_all.conf 拷贝到sphinx服务器/usr/local/webserver/sphinx/etc/

配置文件注意事项

需要PHP支持的配置:script/sphinx/sphinx.conf

#!/usr/bin/php
<?php
//MYSQL参数和WEB服务器中的config/production/database.php MYSQL参数一致
//如果有主从配置,请选择一个从服务器的配置
$sourceConfig = array(
    "type     = mysql",
    "sql_host = 127.0.0.1",
    "sql_user = root",
    "sql_pass = root",
    "sql_db   = b2b2c",
    "sql_port = 3306",
);
$sourceConfig = implode("  \n", $sourceConfig);

$files = scandir(dirname(__FILE__) . "/conf.d/");
foreach($files as $key => $file)
{
    if($file != "." && $file != ".." && preg_match('/.conf$/iU', $file) )
    {
        $tmpFile = dirname(__FILE__) . "/conf.d/$file";
        $fileContents = file_get_contents($tmpFile);
        echo str_replace('{sourcesConfig}', $sourceConfig, $fileContents);
    }
}
?>

indexer
{
    mem_limit        = 80M
}

searchd
{
    listen            = 9312
    listen          = 9306:mysql41
    log                = /usr/local/var/log/searchd.log //日志文件需要读写权限
    query_log        = /usr/local/var/log/query.log //日志文件需要读写权限
    read_timeout    = 5
    client_timeout    = 300
    max_children    = 30
    pid_file        = /usr/local/var/log/searchd.pid //守护进程文件需要读写权限
    seamless_rotate    = 1
    preopen_indexes    = 0
    unlink_old        = 1
    mva_updates_pool= 1M
    max_packet_size    = 8M
    max_filters        = 256
    max_filter_values= 4096
    dist_threads = 2
}

conf.d/*中具体索引配置说明,不可直接使用

source sysitem_item
{
    {sourcesConfig}
    以下省略N个字......
}

index  sysitem_item
{
    source          = sysitem_item
    path            = /usr/local/var/data/sysitem_item #商品搜索索引文件需要读写权限
    docinfo            = extern

    以下省略N个字......
}

sphinx原生支持的配置文件:script/sphinx/sphinx_all.conf 以下为部分注意说明,不可直接使用

source sysitem_item
{
    #MYSQL参数和WEB服务器中的config/production/database.php MYSQL参数一致
    #如果有主从配置,请选择一个从服务器的配置
    type     = mysql
    sql_host = 127.0.0.1
    sql_user = root
    sql_pass = root
    sql_db   = b2b2c
    sql_port = 3306

    以下省略N个字......
}

index  sysitem_item
{
    source          = sysitem_item
    path            = /usr/local/var/data/sysitem_item #商品搜索索引文件需要读写权限
    docinfo            = extern

    以下省略N个字......
}

indexer
{
    mem_limit        = 80M
}

searchd
{
    listen            = 9312
    listen          = 9306:mysql41
    log                = /usr/local/var/log/searchd.log //日志文件需要读写权限
    query_log        = /usr/local/var/log/query.log //日志文件需要读写权限
    read_timeout    = 5
    client_timeout    = 300
    max_children    = 30
    pid_file        = /usr/local/var/log/searchd.pid //守护进程文件需要读写权限
    seamless_rotate    = 1
    preopen_indexes    = 0
    unlink_old        = 1
    mva_updates_pool= 1M
    max_packet_size    = 8M
    max_filters        = 256
    max_filter_values= 4096
    dist_threads = 2
}

1.1.3. 启动sphinx

建立索引indexer --all --rotate

Sphinx 2.3.1-id64-beta (r4926)
Copyright (c) 2001-2015, Andrew Aksyonoff
Copyright (c) 2008-2015, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/usr/local/webserver/sphinx/etc/sphinx.conf'...
indexing index 'sysitem_item'...
collected 395 docs, 0.1 MB
collected 3177 attr values
sorted 0.0 Mvalues, 100.0% done
sorted 0.0 Mhits, 100.0% done
total 395 docs, 87472 bytes
total 0.179 sec, 487168 bytes/sec, 2199.91 docs/sec
total 7133 reads, 0.004 sec, 0.0 kb/call avg, 0.0 msec/call avg
total 15 writes, 0.000 sec, 35.3 kb/call avg, 0.0 msec/call avg

执行searchd开启sphinx, 如果要停止sphinx执行命令searchd --stop

Sphinx 2.3.1-id64-beta (r4926)
Copyright (c) 2001-2015, Andrew Aksyonoff
Copyright (c) 2008-2015, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/usr/local/webserver/sphinx/etc/sphinx.conf'...
listening on all interfaces, port=9312
listening on all interfaces, port=9306
precaching index 'sysitem_item'
precached 1 indexes in 0.002 sec

使用MYSQL协议查看sphinx索引表

[wuwei@elrond bin]$ mysql -P9306 -h127.0.0.1
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 2.3.1-id64-beta (r4926)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show tables;
+--------------------+-------------+
| Index              | Type        |
+--------------------+-------------+
| syscategory_brand  | local       |
| sysitem_item       | distributed |
| sysitem_item_delta | local       |
| sysitem_item_merge | local       |
+--------------------+-------------+
4 rows in set (0.00 sec)

定时更新sphinx索引

* * * * * /usr/local/webserver/sphinx/brand_index.sh
*/1 * * * * /usr/local/webserver/sphinx/delta_index.sh
30 2 * * * /usr/local/webserver/sphinx/main_index.sh

brand_index.sh脚本

#!/bin/bash

#需要修改以下三行数据 sphinx的安装目录、sphinx的配置目录、log的存放目录
sphinx_path="/usr/bin"
sphinx_config_path="/etc/sphinxsearch"
log_path="/usr/local/var/log"

"${sphinx_path}/indexer" syscategory_brand --config "${sphinx_config_path}/sphinx.conf" --rotate

delta_index.sh脚本

#!/bin/bash

#需要修改以下三行数据 sphinx的安装目录、sphinx的配置目录、log的存放目录
sphinx_path="/usr/bin"
sphinx_config_path="/etc/sphinxsearch"
log_path="/usr/local/var/log"

"${sphinx_path}/indexer"  sysitem_item_delta --config "${sphinx_config_path}/sphinx.conf" --rotate
sleep 1
"${sphinx_path}/indexer" --merge sysitem_item_merge sysitem_item_delta --config "${sphinx_config_path}/sphinx.conf" --rotate >> "${log_path}/item_delta_index.log"

main_index.sh脚本

#!/bin/bash

#需要修改sphinx的安装目录
sphinx_path="/usr/bin"

"${sphinx_path}/indexer" --all --rotate

注意:目前为60秒更新一次品牌索引,每天凌晨2:30更新全部索引,可以手动执行更新索引查看全部更新索引的时间是否在可以接受, 如果在不可接受的情况下,可进行配置增量索引。并且可配置MYSQL从服务器,用于sphinx更新索引使用, 商品增量索引已配置好,配置文件为conf.d/item.conf

1.2. WEB服务器:安装scws

如果是多台web服务器则,每台web服务器都需要安装scws

scws下载官网:http://www.xunsearch.com/scws/download.php

scws的git地址:https://github.com/hightman/scws

scws的API地址:https://github.com/hightman/scws/blob/master/API.md

1.2.1. 安装scws

wget http://www.xunsearch.com/scws/down/scws-1.2.3.tar.bz2
tar xvf scws-1.2.3.tar.bz2
cd scws-1.2.3
# --prefix=<scws的安装目录>
./configure --prefix=/usr/local/scws
make && make install

1.2.2. 安装scws的XDB 词典文件

# XDB 格式的词典文件,可用于 SCWS-1.x.x 和 PSCWS4,不可用于 PSCWS23。此为通用词典文件,定制词典或其它服务请查看服务支持页面。
# 将下载的压缩包解压,并将解压得到的 dict.utf8.xdb 放入 /usr/local/scws/etc 目录中
cd /usr/local/scws/etc/
wget http://www.xunsearch.com/scws/down/scws-dict-chs-utf8.tar.bz2
tar xvf scws-dict-chs-utf8.tar.bz2

1.2.3. 安装PHP的scws扩展(回到scws源目录中scws-1.2.3)

cd phpext/
/usr/local/webserver/php/bin/phpize
./configure --with-scws=/usr/local/scws/ --with-php-config=/usr/local/php/bin/php-config
make && make install

配置php.ini,在php.ini中添加如下代码

[scws]
extension = scws.so
scws.default.charset = utf8
;以下的的路径就是编译时的--with-scws的值
scws.default.fpath = /usr/local/scws/

1.3. BBC中配置sphinx

BBC中使用sphinx的配置放在config/search.php

config/search.php复制到config/production/search.php

<?php
 /*
 |--------------------------------------------------------------------------
 | sphinx 配置
 |--------------------------------------------------------------------------
 |
 | 对应原系统: SPHINX_SERVER_HOST SPHINX_PCONNECT
 |
 */
 'sphinx' => array (
     'host' => '127.0.0.1:9306',//指定sphinx服务器的ip和端口
     'pconnect' => true,  // 是否启用sphinx持续连接
 ),

 /*
 |--------------------------------------------------------------------------
 | scws 配置
 |--------------------------------------------------------------------------
 |
 | 对应原系统: SCWS_DICT SCWS_RULE
 | 如果是集群部署|词典路径需一致|或者词典放在同步目录里面调用
 |
 */
  'scws' => array(
    'dict' => '/usr/local/scws/etc/dict.utf8.xdb',
    'rule' => '/usr/local/scws/etc/rules.utf8.ini',
  ),

1.4. 配置后运营平台中开启sphinx搜索

系统-》搜索配置-》搜索引擎管理

查看状态,如果状态为连接异常
    1 config/production/search.php中的sphinx服务器ip和端口是否正确
    2 到sphinx服务器中查看是否开启,到sphinx执行下 mysql -P9306 -h127.0.0.1
    3 查看sphinx服务器iptables是否屏蔽了web服务器

如果显示正常则可以点击 启用 则表示正常使用了sphinx

系统-》搜索配置-》索引管理

启用了sphinx 则可以在索引管理中看到当前sphinx中有哪些索引表和对应的索引字段,对应的索引字段类型

results matching ""

    No results matching ""