Merge branch 'master' of https://github.com/hlgr360/hlgr360.github.io into hlgr360-master
Conflicts: _posts/2016-01-11-log-aggregation.md _posts/2016-01-18-fluentd-log-parsing.md
This commit is contained in:
commit
6f3a81c0f7
49 changed files with 20653 additions and 533 deletions
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: We are live or How to start a developer blog
|
||||
subtite: A short introduction
|
||||
description: How to start our developer blog
|
||||
subtitle: The 'Hello World' Post
|
||||
category: general
|
||||
tags: [cto, culture]
|
||||
author: Holger
|
||||
author_email: holger.reinhardt@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
So how does one start a developer blog? It is pretty intimidating to look at blank editor (BTW, I use [Mou](http://25.io/mou/) to write this post ;) and think about some witty content, some heading which would rope you in and make you want to read what we have to say. But why should you? And who are we anyhow?
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: Notes from OSCON Europe 2015
|
||||
subtite:
|
||||
description: Notes from OSCON Europe 2015
|
||||
title: OSCON Europe 2015
|
||||
subtitle: Notes from OSCON Europe 2015
|
||||
category: conference
|
||||
tags: [open-source]
|
||||
author: Marco
|
||||
author_email: marco.seifried@haufe-lexware.com
|
||||
author_email: marco.seifried@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
This is a personal and opinionated summary of my attendance of the [OSCON](http://conferences.oreilly.com/oscon/open-source-eu-2015) conference this year.
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: The beginnings of our API Journey
|
||||
subtite: Intro to our API style guide
|
||||
description: Got API?
|
||||
subtitle: Intro to our API Style Guide
|
||||
category: api
|
||||
tags: [api]
|
||||
author: Holger
|
||||
author_email: holger.reinhardt@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
Before joining [HaufeDev](http://twitter.com/HaufeDev) I was fortunate to work in the [API Academy](http://apiacademy.co) consultancy with some of the smartest guys in the API field. So it was quite predictable that I would advocate for APIs as one of the cornerstones in our technology strategy.
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: Notes from DevOpsCon 2015 - Part 1
|
||||
subtite:
|
||||
description: Notes from DevOpsCon 2015
|
||||
title: Impressions from DevOpsCon 2015
|
||||
subtitle: Notes from DevOpsCon 2015
|
||||
category: conference
|
||||
tags: [docker, devops]
|
||||
author: Rainer
|
||||
author_email: rainer.zehnle@haufe-lexware.com
|
||||
author_email: rainer.zehnle@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
Elias Weingaertner, Helmut Strasser and I attended [DevOpsCon](http://devopsconference.de/de/) in Munich from 23th - 25th November 2015.
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: Impressions from DockerCon 2015 - Part 1
|
||||
subtite: Insights, Outlooks and Inbetweens
|
||||
description: Notes from DevOpsCon 2015
|
||||
subtitle: Insights, Outlooks and Inbetweens
|
||||
category: conference
|
||||
tags: [docker, security]
|
||||
author: ThomasSc
|
||||
author_email: thomas.schuering@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
Once upon a time in a ~~galaxy~~ container far, far away ... We, a bunch or ~~rebels~~ Haufe-employees, entered the halls of container - wisdom: DockerCon EU 2015 in Barcelona, Spain. Hailing from different departments and locations (Freiburg AND Timisoira, CTO's, ICT, DevOps, ...), the common goal was to learn about the current state of Docker, the technology behind it and its evolving eco-system.
|
||||
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: APIdays Paris - From Philosophy to Technology and back again
|
||||
subtite: A biased report from APIdays global in Paris
|
||||
description: APIdays Paris - From Philosophy to Technology and back again
|
||||
subtitle: A biased report from APIdays Global in Paris
|
||||
category: conference
|
||||
tags: [api]
|
||||
author: Martin Danielsson
|
||||
author_email: martin.danielsson@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
Having just recently come home again from the [APIdays](http://www.apidays.io) conference in Paris (Dec 8-9th 2015), memories are still quite fresh. It was a crowded event, the first day hosting a whopping 800 API enthusiasts, ranging from the geekiest of the geeks to a fair amount of suited business people, showing that talking about APIs is no longer something just the most avantgardist of companies, the most high tech of the tech companies, spend their time with. *Au contraire* (we were in Paris after all), APIs are mainstream, and they contribute to the advancing of collaboration and automatization of the (digital) world as such.
|
||||
|
|
|
@ -1,15 +1,15 @@
|
|||
---
|
||||
layout: post
|
||||
title: Impressions from DockerCon 2015 - Part 2
|
||||
subtitle: Barcelona 16. + 17.11.2015
|
||||
description: my highlights and picks from DockerCon 2015
|
||||
subtitle: Highlights and picks from DockerCon 2015
|
||||
category: conference
|
||||
tags: [docker]
|
||||
author: Peter Frey
|
||||
author_email: peter.frey@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
There are already some weeks since last [DockerCon 2015](http://europe-2015.dockercon.com/), and I want to share some ideas and thoughts that I took from there. The conference took place on November 16th and 17th this year in Barcelona, and was attended by over 1000 participants from Europe and beyond. The estimation is based on the size of the large [forum auditorium] (http://www.ccib.es/spaces/forum-auditorium) that may take up 3.140 persons that got filled for the three plenary sessions.
|
||||
There are already some weeks since last [DockerCon 2015](http://europe-2015.dockercon.com/), and I want to share some ideas and thoughts that I took from there. The conference took place on November 16th and 17th this year in Barcelona, and was attended by over 1000 participants from Europe and beyond. The estimation is based on the size of the large [forum auditorium](http://www.ccib.es/spaces/forum-auditorium) that may take up 3.140 persons that got filled for the three plenary sessions.
|
||||
|
||||
First of all, some background, although [Docker](https://www.docker.com/what-docker) as a technology or hype or platform - however you conceive it - is in the meantime an already well-known terminology. And a large number of articles have already been published on it in the last two years. Docker was [initially released in 2013](https://en.wikipedia.org/wiki/Docker_\(software\)#History). so it is a relatively new. My first experience with Docker was last year, in Spring 2014, when I was asked to do a prototype implementation for the Haufe editorial system (internally known as HRS). Docker was new, and it was new to me, and I struggled with a lot of does and don'ts when transforming an environment - even the small part choosen for the prototype - that has been grown over years and with heavy data centricity.
|
||||
|
||||
|
@ -19,22 +19,22 @@ So I was excited to visit DockerCon and see how Docker continued to evolve as pl
|
|||
|
||||
The past waves of virtualization are now commodity, has reached IT and is no longer the domain of development as it was years ago, when we started with VMware for development and testing. It is the base for nowadays deployment. Virtualization has many aspects and flavours, but one thing is in common: it is rather heavy weighted to build up a virtualization platform, and using it will cause some performance reduction in comparision to deploying software artifacts directly to physical machines - what was still done for this reason, to have maximum throughput and optimal performance for business. But with virtualization we gain flexibility, beeing able to move a virtualized computing unit on the hardware below, especially from an older system to a newer one without having to rebuild, repackage or deploy anything. And it is already a big, well known industry behind virtualization infrastructure and technology.
|
||||
|
||||
So what is new with Docker? First of all, Docker is *very lightweight*. It fits well in modern Unix enviroments as it bases upon kernel features like CGroups, LXC and more to provide a separation of the runtime environment for the application components from the base os system and drivers and hardware below. But docker is not linux only, there is movement also in the non-Linux part of our world implementing docker and docker related services. Important is: Docker is not about VMs, it is containers. Docker as technology and platform promises to become a radical shift in view. But as I am no authority in this domain, I just refer to a recent article on why Docker is [the biggest disruption in Linux virtualization] (http://www.nextplatform.com/2015/11/06/linux-containers-will-disrupt-virtualization-incumbents/).
|
||||
So what is new with Docker? First of all, Docker is *very lightweight*. It fits well in modern Unix enviroments as it bases upon kernel features like CGroups, LXC and more to provide a separation of the runtime environment for the application components from the base os system and drivers and hardware below. But docker is not linux only, there is movement also in the non-Linux part of our world implementing docker and docker related services. Important is: Docker is not about VMs, it is containers. Docker as technology and platform promises to become a radical shift in view. But as I am no authority in this domain, I just refer to a recent article on why Docker is [the biggest disruption in Linux virtualization](http://www.nextplatform.com/2015/11/06/linux-containers-will-disrupt-virtualization-incumbents/).
|
||||
|
||||
##Docker fundamentals##
|
||||
|
||||
There was one session that made a deep impression on me. It was the session titled ["Cgroups, namespaces and beyond: what are containers made from"] (http://de.slideshare.net/Docker/cgroups-namespaces-and-beyond-what-are-containers-made-from) by Jerome Pettazzoni. Jerome shows how Docker bases on and evolves from Linux features like cgroups and namespaces and LXC (Linux containers). Whereas early docker releases based on LXC, it uses now an own abstraction from underlying OS and Linux kernel called libcontainer. He show containers can be built from out of the box linus features in an impressive demo. The main message I took from this presentation: Docker introduces no overhead in comparision to direct deployment to a linux system, as the mechanism used by Docker are system inherent, it means are also used and in effect when one uses Linux without, as they sit there and are used anywhere. Docker is lightweight, really, and has nearly no runtime overhead, so it should be the natural way to deploy and use non-OS software components on Linux.
|
||||
There was one session that made a deep impression on me. It was the session titled ["Cgroups, namespaces and beyond: what are containers made from"](http://de.slideshare.net/Docker/cgroups-namespaces-and-beyond-what-are-containers-made-from) by Jerome Pettazzoni. Jerome shows how Docker bases on and evolves from Linux features like cgroups and namespaces and LXC (Linux containers). Whereas early docker releases based on LXC, it uses now an own abstraction from underlying OS and Linux kernel called libcontainer. He show containers can be built from out of the box linus features in an impressive demo. The main message I took from this presentation: Docker introduces no overhead in comparision to direct deployment to a linux system, as the mechanism used by Docker are system inherent, it means are also used and in effect when one uses Linux without, as they sit there and are used anywhere. Docker is lightweight, really, and has nearly no runtime overhead, so it should be the natural way to deploy and use non-OS software components on Linux.
|
||||
|
||||
##Docker and Security##
|
||||
|
||||
When I started in 2014, one message from IT was: Docker is insecure, and not ready for production use, we cannot support it. Indeed there are a couple of security issues related to Docker, especially if the application to deploy depends on NFS needed to share configuration data and provide a central pool of storage to be accessed by a multi-node system (as HRS is, for reasons of scaling and load balancing). In a Docker container, you are root, and this implies also root access to underlying services, such as NFS mounted volumes. Nowadays, still true, unfortunately. You will find the discussions in various discussion groups in the internet. For example in ["NFS Security in Docker"] (https://groups.google.com/forum/#!topic/docker-user/baFYhFZp0Uw) and many more.
|
||||
|
||||
But there are be big advances with Docker that may slip into the next versions that are planned. One of them, that I yearn to have, is called user namespace mapping. It was announced at DockerCon in more than one presentation, but I remember from "Understanding Docker Security", presented by two memobers of the Docker team, Nathan McCauley and Diogo Monica. The reason why it is not yet final is that it requires further improvements and testing, and so it is only available in the experimental branch of docker, currently.
|
||||
The announcement can be read here: []"User namespaces have arrived in Docker"] (http://integratedcode.us/2015/10/13/user-namespaces-have-arrived-in-docker). The concept of user namespaces in Linux itself is described in the [linux manpages] (http://man7.org/linux/man-pages/man7/user_namespaces.7.html) and may be supported by a few up-to-date linux kernels. So it is something for the hopefully near future. See also the known restrictions section in [github project 'Experimental: User namespace support'](https://github.com/docker/docker/blob/master/experimental/userns.md).
|
||||
The announcement can be read here: []"User namespaces have arrived in Docker"] (http://integratedcode.us/2015/10/13/user-namespaces-have-arrived-in-docker). The concept of user namespaces in Linux itself is described in the [linux manpages](http://man7.org/linux/man-pages/man7/user_namespaces.7.html) and may be supported by a few up-to-date linux kernels. So it is something for the hopefully near future. See also the known restrictions section in [github project 'Experimental: User namespace support'](https://github.com/docker/docker/blob/master/experimental/userns.md).
|
||||
|
||||
An other progress with container security is the project [Notary and docker content trust](https://github.com/docker/notary). It was briefly presented at DockerCon, and I would have to dive deeper into this topic to say more on it. Interesting news is also support for secure hardware based security. To promote that, every participant in one of the general sessions got a YubiKey 4 Nano device, and its use for two factor authentication with code in a Docker repository was demonstrated in the session. The announcement can be found in ["Yubico Launches YubiKey 4 and Touch-to-Sign Functionality at DockerCon Europe 2015"](http://www.marketwired.com/press-release/yubico-launches-yubikey-4-and-touch-to-sign-functionality-at-dockercon-europe-2015-2073790.htm).
|
||||
More technical information on it can be read in the blog article [Docker Content Trust] (https://blog.docker.com/2015/08/content-trust-docker-1-8/).
|
||||
See also the [InnoQ article] (http://www.infoq.com/news/2015/11/docker-security-containers) and the [presentation](https://blog.docker.com/2015/05/understanding-docker-security-and-best-practices/) from May 2015.
|
||||
More technical information on it can be read in the blog article [Docker Content Trust](https://blog.docker.com/2015/08/content-trust-docker-1-8/).
|
||||
See also the [InnoQ article](http://www.infoq.com/news/2015/11/docker-security-containers) and the [presentation](https://blog.docker.com/2015/05/understanding-docker-security-and-best-practices/) from May 2015.
|
||||
|
||||
##Stateless vs Persistency##
|
||||
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: Howto - Leverage 'Let's Encrypt' on Azure
|
||||
title: Using 'Let's Encrypt' Certificates with Azure
|
||||
subtitle: Create free valid SSL certificates in 20 minutes.
|
||||
description: Howto - Leverage 'Let's Encrypt' on an Azure Ubuntu VM
|
||||
category: howto
|
||||
tags: [security, cloud]
|
||||
author: Martin Danielsson
|
||||
author_email: martin.danielsson@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
[Let's Encrypt](https://letsencrypt.org/) is a new Certificate Authority which has a couple of benefits almost unheard of before: It's free, automated and open. This means you can actually use Let's Encrypt to create real SSL certificates which will be accepted as valid by web browsers and others.
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: Creating the Smartsteuer 'Snap' App
|
||||
subtitle: A backstage report
|
||||
description: A behind the scenes view of the birth of our youngest creation.
|
||||
subtitle: A behind the scenes view of the birth of our youngest creation.
|
||||
category: product
|
||||
tags: [smartsteuer, mobile, custdev]
|
||||
author: Eike Hirsch
|
||||
author_email: eike.hirsch@smartsteuer.de
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
As we at [smartsteuer](https://www.smartsteuer.de) really enjoyed how our [Smartsteuer Smartphone App](https://www.smartsteuer.de/online/steuererklaerung-online/#smartphone) was imagined and eventually created, I thought it might be fun to write about it. This blog post is not that much technical but describes our journey to a product which (hopefully) will create value for our customers.
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: DevOpsCon 2015 - Is it really about the tools?
|
||||
subtite: A opinionated commentary by Elias Weingaertner
|
||||
description: My personal findings from DevOpsCon 2015 in Munich
|
||||
subtitle: My opinionated findings from DevOpsCon 2015 in Munich
|
||||
category: conference
|
||||
tags: [devops, microservice]
|
||||
author: Elias Weingaertner
|
||||
author_email: elias.weingaertner@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
Two weeks ago, I attended the DevOp Conference (DevOpsCon) in Munich. As expected, it turned out to be the Mecca for Docker fans and Microservice enthusiasts. While I really enjoyed the conference, I drew two somehow controversial conclusions that are open for debate:
|
||||
|
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
layout: post
|
||||
title: Howto - Log Aggregation with Fluentd, Elasticsearch and Kibana
|
||||
subtite: A short introduction
|
||||
description: Introduction to log aggregation using Fluentd, Elasticsearch and Kibana
|
||||
title: Log Aggregation with Fluentd, Elasticsearch and Kibana
|
||||
subtitle: Introduction to log aggregation using Fluentd, Elasticsearch and Kibana
|
||||
category: howto
|
||||
tags: [devops, docker, logging]
|
||||
author: Doru Mihai
|
||||
author_email: doru.mihai@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
With an increasing number of systems decoupled and scattered throughout the landscape it becomes increasingly difficult to track and trace events across all systems.
|
||||
|
|
|
@ -1,14 +1,16 @@
|
|||
---
|
||||
layout: post
|
||||
title: Howto - Configuring Log Parsing with Fluentd
|
||||
subtite: Approaches to log parsing
|
||||
description: Description of a couple of approaches to designing your fluentd configuration.
|
||||
title: Better Log Parsing with Fluentd
|
||||
subtitle: Description of a couple of approaches to designing your fluentd configuration.
|
||||
category: howto
|
||||
tags: [devops, logging]
|
||||
author: Doru Mihai
|
||||
author_email: doru.mihai@haufe-lexware.com
|
||||
header-img: "images/bg-post.jpg"
|
||||
---
|
||||
|
||||
# Approaches to log parsing
|
||||
|
||||
When you will start to deploy your log shippers to more and more systems you will encounter the issue of adapting your solution to be able to parse whatever log format and source each system is using. Luckily, fluentd has a lot of plugins and you can approach a problem of parsing a log file in different ways.
|
||||
|
||||
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue