hadoop security No groups available

【摘要】Hadoop NameNode 日志疯狂滚动,每天80G以上,日志中全是类似WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user xxx的报错,说明一下,任务是通过hive提交的。

google了大半天,没有建设性的意见,最后发现了这个链接中有这样一段话:
http://hakunamapdata.com/a-user-having-surprising-troubles-running-more-resource-intensive-hive-queries/

Possible fixes

How this problem could be solved? Obviously, a quick and dirty solution is to create an account on NameNode server for each user who accessing HDFS (directly, or by submitting MapReduce jobs to the cluster). However, for many reasons, you do not want to give everybody an account on NameNode server.

2015-07-14 10:44:23,989 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user sudops
2015-07-14 10:44:23,991 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user sudops
org.apache.hadoop.util.Shell$ExitCodeException: id: sudops:无此用户

	at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
	at org.apache.hadoop.util.Shell.run(Shell.java:182)
	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
	at org.apache.hadoop.util.Shell.execCommand(Shell.java:461)
	at org.apache.hadoop.util.Shell.execCommand(Shell.java:444)
	at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:68)
	at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:45)
	at org.apache.hadoop.security.Groups.getGroups(Groups.java:79)
	at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1054)
	at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.<init>(FSPermissionChecker.java:50)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5464)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkTraverse(FSNamesystem.java:5447)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:2168)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.getFileInfo(NameNode.java:888)
	at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)

u2

Related Posts

提供免费IP查询接口,Json格式返回方便程序调用

本站提供IP查询接口,IP数据来源于APNIC.net,并结合GeoIP和纯真IP库整理而成,查询结果包含国家、省份/城市、运营商、经纬度、IP范围、归属地等;另外为方便在程序中调用特提供了接口调用,返回Json格式;本机接口完全免费,欢迎使用!

国内运营商DNS劫持严重,企业网站尽快转到HTTPS

[摘要] 国内运营商DNS劫持现象真的很严重,今天在家从 r…

You Missed

OpenClaw 完整使用指南:自托管 AI Agent 的架构与实战

  • u2
  • 2月 25, 2026
  • 37 views
OpenClaw 完整使用指南:自托管 AI Agent 的架构与实战

Anthropic 推出 Claude Code Security:AI 驱动的代码安全神器

  • u2
  • 2月 24, 2026
  • 36 views

来自 OWASP 的代码安全检查工具:Dependency-Check,为你消除安全隐患

  • u2
  • 10月 16, 2025
  • 20 views
来自 OWASP 的代码安全检查工具:Dependency-Check,为你消除安全隐患

技术宅如何躲开越来越垃圾的CSDN?

  • u2
  • 4月 11, 2024
  • 10 views
技术宅如何躲开越来越垃圾的CSDN?

mysql连接失败:The driver has not received any packets from the server

  • u2
  • 3月 26, 2024
  • 12 views
mysql连接失败:The driver has not received any packets from the server

几个IT小哥常用的工具挂图

  • u2
  • 3月 22, 2024
  • 4 views