Author Archives: Lance

CentOS 6: Install Samba

Versions

  1. CentOS 6.4
  2. Samba 3.6.9
  3. Windows 7

Install

Configure

  1. Add linux user for samba access:
    Note
    Substitute $SAMBA_USER for samba username

  2. Set linux user password:

  3. Add linux user to samba (use same password)

  4. Optional: Add a path to share and change ownership to samba user:
    Note
    Substitute $PATH with the absolute path to the directory to share

    Note
    home directories are shared by the default samba config
  5. Optional: Set selinux context for directory to “samba_share_t” to permit sharing:

  6. Optional: Set selinux bool to make home directories shareable:

  7. Edit /etc/samba/smb.conf and change the workgroup to match the Windows 7 workgroup:
    Note
    Substitute $WG with the name of the workgroup

    Note
    if not configured, Windows 7 defaults to the workgroup name “workgroup”
  8. Add the following to the end of /etc/samba/smb.conf:
    Note
    Substitute $SHARE_NAME with the name of the share

  9. Start services

  10. Start on boot

  11. Open the following ports for samba:

Test

  1. On Windows 7, open command-line and map drive:
    Note
    Substitute $HOSTNAME for the hostname of the samba machine, $PASSWORD with the password of the samba user

    Note

    It may be necessary to run

    to remove active connections and clear cached connection information if you have attempted to connect to the CentOS machine before.

CentOS 6.5: Virtualbox Guest Additions


Versions

  1. CentOS 6.5
  2. Virtualbox 4.3.12

Prerequisites

  1. Install CentOS as Virtualbox Guest

Install Prereqs


Configure

Get the kernel version:

Note

VBoxLinuxAdditions.sh looks for the kernel source at $KERN_DIR/${uname -r}. The cp below is to make the directory structure meet its expectations (the default kernel sources folder includes the minor version)


Install


Verify


Cleanup


Reboot

[DEPRECATED] CentOS 6: Install Datastax Cassandra OpsCenter Community Edition

Versions

  • CentOS 6.4
  • Oracle Java JDK 1.6
  • OpsCenter Community 3.0.2

Prerequisites

Install

1. Edit /etc/yum.repos.d/datastax.repo

2. Install Opscenter Free

Configure

Note: by default opscener will only accept connections from 127.0.0.1:8888. This guide won’t change that setting but it can be changed by editing /etc/opscenter/opscenterd.conf

1. Start opscenter:

4. Start cassandra on boot:

Test

1. Reconnect ssh session and tunnel the default opscenter port (8888):

2. Connect to 127.0.0.1:8888 with a browser
3. Click “Use Existing Cluster”
DataStax OpsCenter - Google Chrome_026
4. Enter a seed cluster node hostname or IP
Note: you can enter a new-line separated list of all nodes, but this is unnecessary
DataStax OpsCenter - Google Chrome_027
5. Click “Save cluster”
DataStax OpsCenter - Google Chrome_028

Sources

Why Scala?

Job hunting as of late. I’ve talked to an endless stream of recruiters, HR managers, tech managers, engineers over the last couple of weeks. They always want to know what kind of work I’m interested in — and for me that’s an easy answer: Scala. I am prioritizing Scala work above all other factors (salary, location, free soda, massages, etc). Everyone always wants to know why I am putting such a heavy emphasis on finding Scala work. Honestly, it’s just that awesome. For the most part, I think I’ve made a good argument for Scala, but I’m always struck with that post-conversation moment of clarity that usually goes something like, “Oh man! I forgot to mention that Scala also does X and Y *and* Z!”. Over the next week, I’m going to make a series of posts that iterate every reason I can think of that someone might find value in learning and adopting Scala. Hopefully, doing this will etch these reasons into my mind for easy recall the next time somebody asks. As a bonus, I’ll end the series with a post to explore the issues, limitations and “why-not”‘s of Scala that I’ve encountered so far.

Post Series:

  1. Code preemptively and eliminate entire classes of errors at compile-time
    1. Use Option to eliminate null
    2. Use immutable variables and immutable collections to eliminate syncronized
    3. Use actors to achieve race and deadlock free concurrency
    4. Leverage static-type checking to encode type-state at compile-time to eliminate issues such as taint or locking
  2. Leverage new language features that improve on Java:
    1. Get easy concurrency using Scala collection libraries’ parallel collections and methods
    2. Type less and grok faster with type-inferencing
    3. Imagine interfaces but with default method implementations
    4. Get a value from all statements, including if and try
    5. Auto-generate bean-style objects using case classes
    6. “Switch” on anything — not just int
    7. Restrict the allowed types of a generic class
    8. Reduce complexity with type-aliases
    9. Put more than one class in a file
    10. Import packages where it makes sense
    11. Easily refer to classes with the same name without typing the entire package name
    12. Type XML literals straight into your code
  3. Build on the best of Java’s features:
    1. Don’t give up static-typing and auto-completing IDEs
    2. Keep all the benefits of the JVM – portability, JIT, garbage collection
    3. Call any existing Java library from Scala with no extra work
    4. Call new Scala code from Java with only a little of bit of work
  4. Learn functional programming AND be productive at the same time:
    1. Code functionally as you learn
    2. Code immutably by default but be mutable when you need it
    3. Use functions as first-class objects
    4. Program with functional idioms to embed semantics into your code
    5. Use currying and compose partial functions
    6. Delay computations until when you need them
    7. Create type-classes using implicit class
    8. Use for-comprehensions and monads to do things you never imagined possible
    9. Fuse functional-programming and object-oriented programming
  5. Extend Scala with your own DSL:
    1. Override operators
    2. Use implicits to automatically convert types
    3. Stop using parentheses and periods
  6. Discover the Scala community
    1. Open-source libraries
    2. Frameworks
  7. Why not Scala?
    1. Binary incompatibility
    2. Optimization, tools and support are still maturing
    3. Production may suffer while learning
    4. Functional code has a learning curve
    5. Functional code can be cryptic
    6. Functional code has performance penalties
    7. Hard to find good Scala developers
    8. Real-time computing

[DEPRECATED] CentOS 6: Configure PAM authentication using Keroberos Tickets

Versions

  • CentOS 6.4

Prerequsities

Install

Configure

Test

1. Add Kerberos authenticated user as an admin

2. Verify by logging in as the user.

Sources

[DEPRECATED] When WSO2 Data Services Server 3.0.1 and Cassandra 1.1 CQL 2 JDBC driver — doesn’t support BLOB

Attempted to use the following CQL 2 schema with two BLOBs:

With the following DSS configuration:

But received the following error:

[DEPRECATED] WSO2 Data Services Server 3.0.1 adding input operation parameter that maps to a query input BLOB parameter results in NullPointerException

Config:

Results in the following error:

If <with-param name="ekg" query-param="ekg"/> is removed then this error is not encountered.

[DEPRECATED] WSO2 Data Services Server 3.0.1 doesn’t work with Cassandra CQL 3

WSO2 Data Services Server 3.0.1 comes with support for an unspecified Cassandra version. However, in their documentation they call out 1.1.X. After much frustration and many NullPointerExceptions from WSO2, I discovered that while Cassandra 1.1 supports CQL 3, it defaults to CQL 2 in the JDBC driver. I also noticed that despite general JDBC support in DSS, the Cassandra JDBC driver is bundled with DSS 3.0.1 and appears to have been customized by WSO2 to support specifically Cassandra 1.1 and DSS 3.0.1.

Tonight I attempted to use the Cassandra JDBC 1.2 driver with WSO2 DSS’s support for “generic” JDBC data sources. I was able to get DSS to connect to the Cassandra server, but running even a simple SELECT query failed with a NullPointerException. To meet my current objectives, I’ve rolled back Cassandra to 1.1 to be compatible with WSO2 DSS 3.0.1 and reverted to using CQL 2. This is a bit of a disappointment since CQL 2 doesn’t seem to have any support for leveraging Cassandra’s “schema-free” features that CQL 3 does (map, set, list, etc).

[DEPRECATED] CentOS 6: Install Single-node Hadoop from Cloudera CDH

Overview

Guide for setting up a single-node Hadoop on CentOS using the Cloudera CDH repository.

Versions

  • CentOS 6.4
  • Oracle Java JDK 1.6
  • CDH 4
  • Hadoop 0.2

Prerequisties

Install

1. Download the yum repo file:

2. Install

Configure

1. Format the name node

Output:

2. Start namenode/datanode services

3. Optional: Start services on boot

4. Create directories

5. Create map/reduce directories

6. Start map/reduce services

7. Optional: Start services on boot

8. Optional: Create a home directory on the hdfs for the current user

9. Edit /etc/profile.d/hadoop.sh

10. Load into session

Test

1. Get a directory listing from hadoop hdfs

Output:

Note: results will vary based on user directories created

2. Navigate browser to http://<hostname>:50070
Hadoop NameNodeĀ localhost:8020 - Google Chrome_024

4. Navigate browser to http://<hostname>:50030
localhost Hadoop Map-Reduce Administration - Google Chrome_023

3. Run one of the examples

Output:

Sources

JVM memory settings

Versions

  • Java Oracle JDK 1.6.X

Heap

On program startup, Java allocates a certain amount of memory for the heap. This memory is used to fulfill requests for memory allocation by the program or libraries it calls. The Java heap space also has an upper limit that if reached, results in the java.lang.OutOfMemoryError [3] being thrown. Generally once running, there is no action a program can take to resolve this error. Instead, either set the maximum heap space higher when running the program or use a Java memory profiler to analyze memory usage (there might be a memory leak). It is also possible to redesign the algorithm to use the disk as a temporary swap space, but this should be avoided unless absolutely necessary.

Java allows setting the minimum and maximum heap settings as follows: [1]

  • -Xms${MIN_VALUE}
  • -Xmx${MAX_VALUE}

Values are specified as multiples of 1024 bytes and must be greater than 1 MB. Append ‘k’ for kilobytes, ‘m’ for megabytes or ‘g’ for gigabytes.

Examples:

  • -Xms16k
  • -Xmx32m
  • -Xmx2g

If these values are not specified at program startup, Java will automatically determine the values to scale with the available system memory. [2]

PermGen

Java uses a separate pre-allocated memory region, called the PermGen, to store class files, string constants and other resources located in jar files. With the exception of very large programs, the default value for PermGen is sufficient. Issues with PermGen are more common when running an application server such as GlassFish or Tomcat that loads jar files on-demand. To fix issues with PermGen, simply raise the default value for the PermGen space. Note that it is possible to create a classloader leak [5] with on-demand jar file loading.

  • -XX:MaxPermSize

Example: -XX:MaxPermSize=128m

Application Server Tuning

It is recommended to run application servers such as GlassFish or Tomcat with the following two settings to optimize garbage collection of unused jar resources in PermGen when an application is undeployed.[7]

  • -XX:+CMSClassUnloadingEnabled
  • -XX:+CMSPermGenSweepingEnabled

Sources

  1. http://docs.oracle.com/javase/6/docs/technotes/tools/windows/java.html
  2. http://docs.oracle.com/javase/6/docs/technotes/guides/vm/gc-ergonomics.html
  3. http://docs.oracle.com/javase/6/docs/api/java/lang/OutOfMemoryError.html
  4. http://stackoverflow.com/questions/37335/how-to-deal-with-java-lang-outofmemoryerror-java-heap-space-error-64mb-heap
  5. http://frankkieviet.blogspot.ca/2006/10/classloader-leaks-dreaded-permgen-space.html
  6. http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
  7. http://stackoverflow.com/questions/88235/dealing-with-java-lang-outofmemoryerror-permgen-space-error