Useful Hive LLAP debug info

https://community.hortonworks.com/articles/149896/llap-debugging-overview-logs-uis-etc.html

Advertisements

How to fix Falcon service not started issue in HDP2.5+

FROM: https://community.hortonworks.com/questions/77600/faclon-web-ui-failing-with-http-503-service-unavai.html

If you installed HDP 2.5 or later, you must manually install the Berkeley DB file required for Falcon. Below are the instructions for Ambari installations. See if this helps.

1. Download the required Berkeley DB implementation file.

wget –O je-5.0.73.jar http://search.maven.org/remotecontent?filepath=com/sleepycat/je/5.0.73/je-5.0.73.jar

2. Log in to the Ambari server with administrator privileges.

su – root

3. Copy the file to the Ambari server share folder.

cp je-5.0.73.jar /usr/share/

4. Set permissions on the file to owner=read/write, group=read, other=read.

chmod 644 /usr/share/je-5.0.73.jar

5. Configure the Ambari server to use the Berkeley DB driver.

ambari-server setup –jdbc-db=bdb –jdbc-driver=/usr/share/je-5.0.73.jar

6. Restart the Ambari server.

ambari-server restart

7. Restart the Falcon service from the Ambari UI.

You need to have administrator privileges in Ambari to restart a service.

a) In the Ambari web UI, click the Services tab and select the Falcon service in the left Services pane.

b) From the Falcon Summary page, click Service Actions > Restart All.

c) Click Confirm Restart All.

When the service is available, the Falcon status displays as Started on the Summary page.

How to disable JVM large page support

from http://www.oracle.com/technetwork/java/javase/7u51-relnotes-2085002.html

Area: hotspot/gc
Synopsis: Crashes due to failure to allocate large pages.

On Linux, failures when allocating large pages can lead to crashes. When running JDK 7u51 or later versions, the issue can be recognized in two ways:

  • Before the crash happens one or more lines similar to this will have been printed to the log:os::commit_memory(0x00000006b1600000, 352321536, 2097152, 0) failed;
    error='Cannot allocate memory' (errno=12); Cannot allocate large pages, falling back to regular pages
  • If a hs_err file is generated it will contain a line similar to this:Large page allocation failures have occurred 3 times

The problem can be avoided by running with large page support turned off, for example by passing the "-XX:-UseLargePages" option to the java binary.


If you cannot set this on your command line, you can set this value to JAVA_TOOL_OPTIONS environment variable: export JAVA_TOOL_OPTIONS=”-XX:-UseLargePages”

How to install VirtualBox GuestAddition on a guiless CentOs

https://www.if-not-true-then-false.com/2010/install-virtualbox-guest-additions-on-fedora-centos-red-hat-rhel/

  1. sudo yum update kernel* -y
  2. mkdir /media/cdrom
  3. mount -r /dev/cdrom /media/cdrom
  4. sudo rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
  5. sudo yum install gcc kernel-devel kernel-headers dkms make bzip2 perl -y
  6. KERN_DIR=/usr/src/kernels/uname -r
  7. export KERN_DIR
  8. cd /media/cdrom
  9. ./VBoxLinuxAdditions.run
  10. reboot

How to completely remove docker on windows 10

From: https://success.docker.com/KBase/How_to_completely_remove_Docker

 

Save the following script in a file with extension of ps1.  Log in powershell console as admin and run the script file.

$ErrorActionPreference = “SilentlyContinue”

kill -force -processname ‘Docker for Windows’, com.docker.db, vpnkit, com.docker.proxy, com.docker.9pdb, moby-diag-dl, dockerd

try {
./MobyLinux.ps1 -Destroy
} Catch {}

$service = Get-WmiObject -Class Win32_Service -Filter “Name=’com.docker.service'”
if ($service) { $service.StopService() }
if ($service) { $service.Delete() }
Start-Sleep -s 5
Remove-Item -Recurse -Force “~/AppData/Local/Docker”
Remove-Item -Recurse -Force “~/AppData/Roaming/Docker”
if (Test-Path “C:\ProgramData\Docker”) { takeown.exe /F “C:\ProgramData\Docker” /R /A /D Y }
if (Test-Path “C:\ProgramData\Docker”) { icacls “C:\ProgramData\Docker\” /T /C /grant Administrators:F }
Remove-Item -Recurse -Force “C:\ProgramData\Docker”
Remove-Item -Recurse -Force “C:\Program Files\Docker”
Remove-Item -Recurse -Force “C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Docker”
Remove-Item -Force “C:\Users\Public\Desktop\Docker for Windows.lnk”
Get-ChildItem HKLM:\software\microsoft\windows\currentversion\uninstall | % {Get-ItemProperty $_.PSPath} | ? { $_.DisplayName -eq “Docker” } | Remove-Item -Recurse -Force
Get-ChildItem HKLM:\software\classes\installer\products | % {Get-ItemProperty $_.pspath} | ? { $_.ProductName -eq “Docker” } | Remove-Item -Recurse -Force
Get-Item ‘HKLM:\software\Docker Inc.’ | Remove-Item -Recurse -Force
Get-ItemProperty HKCU:\software\microsoft\windows\currentversion\Run -name “Docker for Windows” | Remove-Item -Recurse -Force
#Get-ItemProperty HKCU:\software\microsoft\windows\currentversion\UFH\SHC | ForEach-Object {Get-ItemProperty $_.PSPath} | Where-Object { $_.ToString().Contains(“Docker for Windows.exe”) } | Remove-Item -Recurse -Force $_.PSPath
#Get-ItemProperty HKCU:\software\microsoft\windows\currentversion\UFH\SHC | Where-Object { $(Get-ItemPropertyValue $_) -Contains “Docker” }

How to fix error “Font family [‘sans-serif’] not found” when using seaborn in jupyter running in a vagrant centos vm

yum install curl cabextract xorg-x11-font-utils fontconfig
yum install https://downloads.sourceforge.net/project/mscorefonts2/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm

Note: if yum cannot install directly, you can download the rpm file from: https://downloads.sourceforge.net/project/mscorefonts2/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm and install it locally.

How to add multiple database support to Spring application

Define multiple configuration files. In each configuration file, specify the JpaVendorAdapter, datasource, enityManager, entityManagerFactory, and transactionManager. You can set a default configuration file and multiple other configuration files.

A sample goes as below:

Default configuration file:

@Configuration
@EnableTransactionManagement
@EnableAutoConfiguration
@EnableJpaRepositories(
        entityManagerFactoryRef = "entityManagerFactory",
        transactionManagerRef = "transactionManager",
        basePackages = {"bmo.hub.bpm.foundation.appmonitor.db.repo"}
)
public class AppConfig {

    @Autowired
    private JpaVendorAdapter jpaVendorAdapter;

    @Autowired
    private DataSource dataSource;

    @Bean(name = "entityManager")
    public EntityManager entityManager() {
        return entityManagerFactory().createEntityManager();
    }

    @Primary
    @Bean(name = "entityManagerFactory")
    public EntityManagerFactory entityManagerFactory() {
        LocalContainerEntityManagerFactoryBean emf = new LocalContainerEntityManagerFactoryBean();
        emf.setDataSource(dataSource);
        emf.setJpaVendorAdapter(jpaVendorAdapter);
        emf.setPackagesToScan("bmo.hub.bpm.foundation.appmonitor.db.entity");
        emf.setPersistenceUnitName("default");
        emf.afterPropertiesSet();

        return emf.getObject();
    }

    @Bean(name = "transactionManager")
    public PlatformTransactionManager transactionManager() {
        JpaTransactionManager tm = new JpaTransactionManager();
        tm.setEntityManagerFactory(entityManagerFactory());

        return tm;
    }
}

Another configuration file:

@Configuration
@EnableTransactionManagement
@EnableJpaRepositories(
        entityManagerFactoryRef = "mortgageEntityManagerFactory",
        transactionManagerRef = "mortgageTransactionManager",
        basePackages = {"bmo.hub.bpm.foundation.appmonitor.db.repo.mortgage"}
)
public class MortgageConfig {

    @Autowired
    private JpaVendorAdapter jpaVendorAdapter;

//    @Autowired
//    private DataSource dataSource;
    @Value("${spring.datasource.mortgage.url}")
    private String databaseUrl;

    @Value("${spring.datasource.mortgage.username}")
    private String username;

    @Value("${spring.datasource.mortgage.password}")
    private String password;

    @Value("${spring.datasource.mortgage.driver-class}")
    private String driverClass;

    @Bean(name = "mortgageEntityManager")
    public EntityManager entityManager() {
        return entityManagerFactory().createEntityManager();
    }

    public DataSource dataSource() {
        DriverManagerDataSource dataSource = new DriverManagerDataSource(databaseUrl, username, password);
        dataSource.setDriverClassName(driverClass);
        return dataSource;
    }

    @Primary
    @Bean(name = "mortgageEntityManagerFactory")
    public EntityManagerFactory entityManagerFactory() {
        LocalContainerEntityManagerFactoryBean emf = new LocalContainerEntityManagerFactoryBean();
        emf.setDataSource(dataSource());
        emf.setJpaVendorAdapter(jpaVendorAdapter);
        emf.setPackagesToScan("bmo.hub.bpm.foundation.appmonitor.db.entity.mortgage");
        emf.afterPropertiesSet();

        return emf.getObject();
    }

    @Bean(name = "mortgageTransactionManager")
    public PlatformTransactionManager transactionManager() {
        JpaTransactionManager tm = new JpaTransactionManager();
        tm.setEntityManagerFactory(entityManagerFactory());

        return tm;
    }
}

And the properties file (application.yml):

selenium:
  grid:
    url: http://localhost:4444/wd/hub

scheduling:
  job:
    cron: 0 0/3 * * * *

spring:
  datasource:
    url: jdbc:oracle:thin:@10.193.61.167:1521:garnish
    username: row_dev
    password: Welcome#1
    driver-class: oracle.jdbc.driver.OracleDriver

    mortgage:
      url: jdbc:oracle:thin:@10.193.46.193:1521:mortgage
      username: mortgage
      password: MORTGAGE
      driver-class: oracle.jdbc.driver.OracleDriver

    hikari:
      connection-timeout: 60000
      maximum-pool-size: 5