开源开发者笔记:DevOps,微服务,分布式,大数据,高可用,区块链,白皮书,算法,黑客,设计模式,面试题。求 star:star:️
Apache Ranger是大数据领域的一个集中式安全管理框架,目的是通过制定策略(policies)实现对Hadoop组件的集中式安全管理。用户可以通过Ranger实现对集群中数据的安全访问。
环境
框架 | 版本 |
---|---|
Ubuntu | 16.04 |
JAVA | OpenJDK-8-jdk |
Python | 2.7 |
Maven | 3.6.3 |
Ranger | 2.0.0 |
Hadoop | 3.1.3 |
Solr | 8.5.2 |
源码编译
环境准备
- 下载 Ranger 源码
wget https://downloads.apache.org/ranger/2.0.0/apache-ranger-2.0.0.tar.gz 或使用镜像 wget https://mirrors.tuna.tsinghua.edu.cn/apache/ranger/2.0.0/apache-ranger-2.0.0.tar.gz 复制代码
- 下载 Maven
wget https://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
复制代码
apt install openjdk-8-jdk apt install python2.7 apt install gcc g++ ln -s /usr/bin/python2.7 /usr/bin/python 复制代码
- 配置环境变量
export MAVEN_HOME=/opt/app/apache-maven-3.6.3 export PATH=$PATH:$MAVEN_HOME/bin export MAVEN_OPTS=-Xmx2048m 复制代码
- 使用 ali maven 源
/opt/app/apache-maven-3.6.2/conf/settings.xml
<mirrors>
<mirror>
<id>nexus-aliyun</id>
<mirrorOf>central</mirrorOf>
<name>Nexus aliyun</name>
<url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>
<mirror>
<id>CN</id>
<name>OSChina Central</name>
<url>http://maven.oschina.net/content/groups/public/</url>
<mirrorOf>central</mirrorOf>
</mirror>
<mirror>
<id>alimaven</id>
<mirrorOf>central</mirrorOf>
<name>aliyun maven</name>
<url>https://maven.aliyun.com/nexus/content/repositories/central/</url>
</mirror>
<mirror>
<id>jboss-public-repository-group</id>
<mirrorOf>central</mirrorOf>
<name>JBoss Public Repository Group</name>
<url>https://repository.jboss.org/nexus/content/groups/public</url>
</mirror>
</mirrors>
复制代码
编译
tar xf apache-ranger-2.0.0.tar.gz cd apache-ranger-2.0.0 mvn clean compile package assembly:assembly install -DskipTests -Drat.skip=true 编译成功打开 target 文件夹 apache-ranger-2.0.0/target# ll total 1605220 drwxr-xr-x 5 root root 4096 Jun 29 11:04 ./ drwxr-xr-x 59 root root 4096 Jun 29 03:20 ../ drwxr-xr-x 2 root root 4096 Jun 29 03:20 antrun/ drwxr-xr-x 2 root root 4096 Jun 29 11:04 archive-tmp/ drwxr-xr-x 3 root root 4096 Jun 29 03:20 maven-shared-archive-resources/ -rw-r--r-- 1 root root 30 Jun 29 11:04 .plxarc -rw-r--r-- 1 root root 248635508 Jun 29 11:01 ranger-2.0.0-admin.tar.gz -rw-r--r-- 1 root root 249666712 Jun 29 11:02 ranger-2.0.0-admin.zip -rw-r--r-- 1 root root 27787895 Jun 29 11:03 ranger-2.0.0-atlas-plugin.tar.gz -rw-r--r-- 1 root root 27832012 Jun 29 11:03 ranger-2.0.0-atlas-plugin.zip -rw-r--r-- 1 root root 31555304 Jun 29 11:04 ranger-2.0.0-elasticsearch-plugin.tar.gz -rw-r--r-- 1 root root 31605841 Jun 29 11:04 ranger-2.0.0-elasticsearch-plugin.zip -rw-r--r-- 1 root root 26638377 Jun 29 10:58 ranger-2.0.0-hbase-plugin.tar.gz -rw-r--r-- 1 root root 26665982 Jun 29 10:58 ranger-2.0.0-hbase-plugin.zip -rw-r--r-- 1 root root 23971392 Jun 29 10:58 ranger-2.0.0-hdfs-plugin.tar.gz -rw-r--r-- 1 root root 23997623 Jun 29 10:58 ranger-2.0.0-hdfs-plugin.zip -rw-r--r-- 1 root root 23825995 Jun 29 10:58 ranger-2.0.0-hive-plugin.tar.gz -rw-r--r-- 1 root root 23854522 Jun 29 10:58 ranger-2.0.0-hive-plugin.zip -rw-r--r-- 1 root root 39930681 Jun 29 10:59 ranger-2.0.0-kafka-plugin.tar.gz -rw-r--r-- 1 root root 39983878 Jun 29 10:59 ranger-2.0.0-kafka-plugin.zip -rw-r--r-- 1 root root 90982883 Jun 29 11:02 ranger-2.0.0-kms.tar.gz -rw-r--r-- 1 root root 91106270 Jun 29 11:02 ranger-2.0.0-kms.zip -rw-r--r-- 1 root root 28380704 Jun 29 10:58 ranger-2.0.0-knox-plugin.tar.gz -rw-r--r-- 1 root root 28411022 Jun 29 10:58 ranger-2.0.0-knox-plugin.zip -rw-r--r-- 1 root root 23940874 Jun 29 11:03 ranger-2.0.0-kylin-plugin.tar.gz -rw-r--r-- 1 root root 23980053 Jun 29 11:03 ranger-2.0.0-kylin-plugin.zip -rw-r--r-- 1 root root 34223 Jun 29 11:02 ranger-2.0.0-migration-util.tar.gz -rw-r--r-- 1 root root 37740 Jun 29 11:02 ranger-2.0.0-migration-util.zip -rw-r--r-- 1 root root 26388071 Jun 29 11:00 ranger-2.0.0-ozone-plugin.tar.gz -rw-r--r-- 1 root root 26421136 Jun 29 11:00 ranger-2.0.0-ozone-plugin.zip -rw-r--r-- 1 root root 40302042 Jun 29 11:04 ranger-2.0.0-presto-plugin.tar.gz -rw-r--r-- 1 root root 40341626 Jun 29 11:04 ranger-2.0.0-presto-plugin.zip -rw-r--r-- 1 root root 22232050 Jun 29 11:02 ranger-2.0.0-ranger-tools.tar.gz -rw-r--r-- 1 root root 22248747 Jun 29 11:02 ranger-2.0.0-ranger-tools.zip -rw-r--r-- 1 root root 42667 Jun 29 11:02 ranger-2.0.0-solr_audit_conf.tar.gz -rw-r--r-- 1 root root 45636 Jun 29 11:02 ranger-2.0.0-solr_audit_conf.zip -rw-r--r-- 1 root root 26964416 Jun 29 11:00 ranger-2.0.0-solr-plugin.tar.gz -rw-r--r-- 1 root root 27010058 Jun 29 11:00 ranger-2.0.0-solr-plugin.zip -rw-r--r-- 1 root root 23952732 Jun 29 11:03 ranger-2.0.0-sqoop-plugin.tar.gz -rw-r--r-- 1 root root 23986050 Jun 29 11:03 ranger-2.0.0-sqoop-plugin.zip -rw-r--r-- 1 root root 4081388 Jun 29 11:03 ranger-2.0.0-src.tar.gz -rw-r--r-- 1 root root 6257752 Jun 29 11:03 ranger-2.0.0-src.zip -rw-r--r-- 1 root root 37230628 Jun 29 10:59 ranger-2.0.0-storm-plugin.tar.gz -rw-r--r-- 1 root root 37268719 Jun 29 10:59 ranger-2.0.0-storm-plugin.zip -rw-r--r-- 1 root root 32772036 Jun 29 11:02 ranger-2.0.0-tagsync.tar.gz -rw-r--r-- 1 root root 32782070 Jun 29 11:02 ranger-2.0.0-tagsync.zip -rw-r--r-- 1 root root 16256778 Jun 29 11:02 ranger-2.0.0-usersync.tar.gz -rw-r--r-- 1 root root 16280886 Jun 29 11:02 ranger-2.0.0-usersync.zip -rw-r--r-- 1 root root 23953849 Jun 29 10:59 ranger-2.0.0-yarn-plugin.tar.gz -rw-r--r-- 1 root root 23992362 Jun 29 10:59 ranger-2.0.0-yarn-plugin.zip -rw-r--r-- 1 root root 5 Jun 29 11:04 version 复制代码
编译时发生错误
- 没安装c++编译环境
[INFO] Unix Native Authenticator .......................... FAILURE [ 1.387 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 08:51 min [INFO] Finished at: 2020-06-030T12:00:43-02:00 [INFO] Final Memory: 96M/420M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:compile (default-compile) on project credValidator: Error executing command line. Exit code:127 -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:compile (default-compile) on project credValidator: Error executing command line. Exit code:127 复制代码
- 没安装python
[INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (generate-version-annotation) on project ranger-util: An Ant BuildException has occured: exec returned: 1 [ERROR] around Ant part ...<exec failonerror="true" executable="python">... @ 4:48 in /data/apache-ranger-2.0.0/ranger-util/target/antrun/build-main.xml [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (generate-version-annotation) on project ranger-util: An Ant BuildException has occured: exec returned: 1 around Ant part ...<exec failonerror="true" executable="python">... @ 4:48 in /data/apache-ranger-2.0.0/ranger-util/target/antrun/build-main.xml 复制代码
安装 ranger-admin
解压
tar zxvf ranger-2.0.0-admin.tar.gz -C /opt/ranger cd /opt/ranger 复制代码
修改 ranger-2.0.0-admin/install.properties
# 数据库连接 SQL_CONNECTOR_JAR=/opt/ranger/mysql-connector-java-8.0.18.jar db_root_user=root db_root_password=root db_host=localhost db_name=ranger db_user=ranger db_password=rangeradmin # 日志审计 audit_store=solr audit_solr_urls=http://localhost:6083/solr/ranger_audits audit_solr_user=solr 复制代码
执行 ranger-2.0.0-admin/setup.sh
进行安装
成功则输出。 install.properties
配置中需要的 solr
、 mysql
、 mysql驱动包
如果没有可参考下面
2020-06-30 07:55:44,211 [I] Checking connection passed.
Installation of Ranger PolicyManager Web Application is completed.
复制代码
执行 ranger-2.0.0-admin/set_globals.sh
配置软连接
执行 ranger-admin start
或者 ranger-2.0.0-admin/ews/ranger-admin-services.sh start
启动服务
浏览器访问 http://ip:6080/
admin/admin

安装 solr
修改 ranger-2.0.0-admin/contrib/solr_for_audit_setup/install.properties
SOLR_INSTALL=true JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 SOLR_DOWNLOAD_URL=https://mirrors.tuna.tsinghua.edu.cn/apache/lucene/solr/8.5.2/solr-8.5.2.tgz SOLR_INSTALL_FOLDER=/data/solr #安装目录,这个随意 SOLR_RANGER_HOME=/data/solr/ranger_audit_server SOLR_RANGER_PORT=6083 #默认端口 SOLR_DEPLOYMENT=standalone #部署模式(单节点部署),还有solrcloud(集群)模式 SOLR_RANGER_DATA_FOLDER=/data/solr/ranger_audit_server/data #数据存放目录 复制代码
执行 ranger-2.0.0-admin/contrib/solr_for_audit_setup/setup.sh
Tue Jun 30 06:58:31 UTC 2020|INFO|Installed Solr in /opt/solr Tue Jun 30 06:58:31 UTC 2020|INFO|Configuring standalone instance Tue Jun 30 06:58:31 UTC 2020|INFO|Copying Ranger Audit Server configuration to /opt/solr/ranger_audit_server Tue Jun 30 06:58:31 UTC 2020|INFO|Creating group solr Tue Jun 30 06:58:31 UTC 2020|INFO|Creating user solr Tue Jun 30 06:58:31 UTC 2020|INFO|Done configuring Solr for Apache Ranger Audit Tue Jun 30 06:58:31 UTC 2020|INFO|Solr HOME for Ranger Audit is /opt/solr/ranger_audit_server Tue Jun 30 06:58:31 UTC 2020|INFO|Data folder for Audit logs is /opt/solr/ranger_audit_server/data Tue Jun 30 06:58:31 UTC 2020|INFO|To start Solr run /opt/solr/ranger_audit_server/scripts/start_solr.sh Tue Jun 30 06:58:31 UTC 2020|INFO|To stop Solr run /opt/solr/ranger_audit_server/scripts/stop_solr.sh Tue Jun 30 06:58:31 UTC 2020|INFO|After starting Solr for RangerAudit, it will listen at 6083. E.g http://weihai-2:6083 Tue Jun 30 06:58:31 UTC 2020|INFO|Configure Ranger to use the following URL http://weihai-2:6083/solr/ranger_audits Tue Jun 30 06:58:31 UTC 2020|INFO| ** NOTE: If Solr is Secured then solrclient JAAS configuration has to be added to Ranger Admin and Ranger Plugin properties Tue Jun 30 06:58:31 UTC 2020|INFO| ** Refer documentation on how to configure Ranger for audit to Secure Solr ########## Done ################### Created file /opt/solr/ranger_audit_server/install_notes.txt with instructions to start and stop ################################### 复制代码
启动/停止 solr
/opt/solr/ranger_audit_server/scripts/start_solr.sh /opt/solr/ranger_audit_server/scripts/stop_solr.sh 复制代码
安装 MySQL
-
下载 mysql 驱动包
https://downloads.mysql.com/archives/c-j/
-
下载 mysql-server
- 官网下载
https://dev.mysql.com/downloads/mysql/ 复制代码
或
- 更新APT源
wget https://dev.mysql.com/get/mysql-apt-config_0.8.15-1_all.de pkg -i mysql-apt-config_0.8.15-1_all.deb apt search mysql-server Sorting... Done Full Text Search... Done auth2db/xenial 0.2.5-2+dfsg-5ubuntu1 all Powerful and eye-candy IDS logger, log viewer and alert generator mysql-server/unknown,now 8.0.20-1ubuntu16.04 amd64 [installed] MySQL Server meta package depending on latest version apt install mysql-server 复制代码
-
配置 mysql
create user 'ranger'@'%' identified by 'rangeradmin';
create database ranger;
grant all privileges on ranger.* to 'ranger'@'%';
set global log_bin_trust_function_creators=TRUE;
SET GLOBAL innodb_lock_wait_timeout=500;
复制代码
原文
https://juejin.im/post/5efb0be66fb9a07e8b21312e
本站部分文章源于互联网,本着传播知识、有益学习和研究的目的进行的转载,为网友免费提供。如有著作权人或出版方提出异议,本站将立即删除。如果您对文章转载有任何疑问请告之我们,以便我们及时纠正。PS:推荐一个微信公众号: askHarries 或者qq群:474807195,里面会分享一些资深架构师录制的视频录像:有Spring,MyBatis,Netty源码分析,高并发、高性能、分布式、微服务架构的原理,JVM性能优化这些成为架构师必备的知识体系。还能领取免费的学习资源,目前受益良多

转载请注明原文出处:Harries Blog™ » Apache Ranger 编译安装