initial commit

This commit is contained in:
Setyo Nugroho 2022-05-13 13:53:52 +07:00
commit 5b6a03b1f9
58 changed files with 2770 additions and 0 deletions

283
.gitignore vendored Normal file
View file

@ -0,0 +1,283 @@
# Created by https://www.toptal.com/developers/gitignore/api/django,pycharm+all
# Edit at https://www.toptal.com/developers/gitignore?templates=django,pycharm+all
### Django ###
*.log
*.pot
*.pyc
__pycache__/
local_settings.py
db.sqlite3
db.sqlite3-journal
media
env/*
# If your build process includes running collectstatic, then you probably don't need or want to include staticfiles/
# in your Git repository. Update and uncomment the following line accordingly.
# <django-project-name>/staticfiles/
### Django.Python Stack ###
# Byte-compiled / optimized / DLL files
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
logs/*
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
# Django stuff:
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
### PyCharm+all ###
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio, WebStorm and Rider
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
# User-specific stuff
.idea/**/workspace.xml
.idea/**/tasks.xml
.idea/**/usage.statistics.xml
.idea/**/dictionaries
.idea/**/shelf
# AWS User-specific
.idea/**/aws.xml
# Generated files
.idea/**/contentModel.xml
# Sensitive or high-churn files
.idea/**/dataSources/
.idea/**/dataSources.ids
.idea/**/dataSources.local.xml
.idea/**/sqlDataSources.xml
.idea/**/dynamic.xml
.idea/**/uiDesigner.xml
.idea/**/dbnavigator.xml
# Gradle
.idea/**/gradle.xml
.idea/**/libraries
# Gradle and Maven with auto-import
# When using Gradle or Maven with auto-import, you should exclude module files,
# since they will be recreated, and may cause churn. Uncomment if using
# auto-import.
# .idea/artifacts
# .idea/compiler.xml
# .idea/jarRepositories.xml
# .idea/modules.xml
# .idea/*.iml
# .idea/modules
# *.iml
# *.ipr
# CMake
cmake-build-*/
# Mongo Explorer plugin
.idea/**/mongoSettings.xml
# File-based project format
*.iws
# IntelliJ
out/
# mpeltonen/sbt-idea plugin
.idea_modules/
# JIRA plugin
atlassian-ide-plugin.xml
# Cursive Clojure plugin
.idea/replstate.xml
# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties
# Editor-based Rest Client
.idea/httpRequests
# Android studio 3.1+ serialized cache file
.idea/caches/build_file_checksums.ser
### PyCharm+all Patch ###
# Ignores the whole .idea folder and all .iml files
# See https://github.com/joeblau/gitignore.io/issues/186 and https://github.com/joeblau/gitignore.io/issues/360
.idea/
# Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-249601023
*.iml
modules.xml
.idea/misc.xml
*.ipr
# Sonarlint plugin
.idea/sonarlint
# End of https://www.toptal.com/developers/gitignore/api/django,pycharm+all
# Created by https://www.toptal.com/developers/gitignore/api/macos
# Edit at https://www.toptal.com/developers/gitignore?templates=macos
### macOS ###
# General
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
# End of https://www.toptal.com/developers/gitignore/api/macos

178
README.md Normal file
View file

@ -0,0 +1,178 @@
# Yuyu
Yuyu provide ability to manage openstack billing by listening to every openstack event. Yuyu is a required component to use Yuyu Dashboard. There are 3 main component in Yuyu: API, Cron, Event Monitor
## Yuyu API
Main component to communicate with Yuyu Dashboard.
## Yuyu Cron
Provide invoice calculation and rolling capabilities that needed to run every month.
## Yuyu Event Monitor
Monitor event from openstack to calculate billing spent.
# System Requirement
- Python 3
- Openstack
- Virtualenv
- Linux environment with Systemd
# Pre-Installation
### Virtualenv
Make sure you installed virtualenv before installing Yuyu
```bash
pip3 install virtualenv
```
### Timezone
Billing is a time sensitive application, please make sure you set a correct time and timezone on you machine.
### Openstack Service Notification
You need to enable notification for this openstack service:
- Nova (nova.conf)
- Cinder (cinder.conf)
- Neutron (neutron.conf)
### Nova
Add configuration below on `[oslo_messaging_notifications]`
```
driver = messagingv2
topics = notifications
```
Add configuration below on `[notifications]`
```
notify_on_state_change = vm_and_task_state
notification_format = unversioned
```
### Cinder & Neutron
Add configuration below on `[oslo_messaging_notifications]`
```
driver = messagingv2
topics = notifications
```
### Kolla Note
If you using Kolla, please add configuration above on all service container. For example on Nova you should put the config on `nova-api`, `nova-scheduler`, etc.
# Installation
Clone the latest source code and put it on any directory you want. Here i assume you put it on `/var/yuyu/`
```bash
cd /var/yuyu/
git clone {repository}
cd yuyu
```
Then create virtualenv and activate it
```bash
virtualenv env --python=python3.8
source env/bin/activate
pip install -r requirements.txt
```
Then create a configuration file, just copy from sample file and modify as your preference.
```bash
cp yuyu/local_settings.py.sample yuyu/local_settings.py
```
Please read [Local Setting Configuration](#local-setting-configuration) to get to know about what configuration you should change.
Then run the database migration
```bash
python manage.py migrate
```
Then create first superuser
```bash
python manage.py createsuperuser
```
## Local Setting Configuration
### YUYU_NOTIFICATION_URL (required)
A Messaging Queue URL that used by Openstack, usually it is a RabbitMQ URL.
Example:
```
YUYU_NOTIFICATION_URL = "rabbit://openstack:password@127.0.0.1:5672/"
```
### YUYU_NOTIFICATION_TOPICS (required)
A list of topic notification topic that is configured on each openstack service
Example:
```
YUYU_NOTIFICATION_TOPICS = ["notifications"]
```
### DATABASE
By default, it will use Sqlite. If you want to change it to other database please refer to Django Setting documentation.
- https://docs.djangoproject.com/en/3.2/ref/settings/#databases
- https://docs.djangoproject.com/en/3.2/ref/databases/
## API Installation
To install Yuyu API, you need to execute this command.
```bash
./bin/setup_api.sh
```
This will install `yuyu_api` service
To start the service use this command
```bash
systemctl enable yuyu_api
systemctl start yuyu_api
```
An API server will be open on port `8182`.
## Event Monitor Installation
To install Yuyu API, you need to execute this command.
```bash
./bin/setup_event_monitor.sh
```
This will install `yuyu_event_monitor` service
To start the service use this command
```bash
systemctl enable yuyu_event_monitor
systemctl start yuyu_event_monitor
```
## Cron Installation
There is a cronjob that needed to be run every month on 00:01 AM. This cronjob will finish all in progress invoice and start new invoice for the next month.
To install it, you can use `crontab -e`.
Put this expression on the crontab
```
1 0 1 * * $yuyu_dir/bin/process_invoice.sh
```
Replace $yuyu_dir with the directory of where yuyu is located. Example
```
1 0 1 * * /var/yuyu/bin/process_invoice.sh
```

0
api/__init__.py Normal file
View file

6
api/apps.py Normal file
View file

@ -0,0 +1,6 @@
from django.apps import AppConfig
class ApiConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'api'

56
api/serializers.py Normal file
View file

@ -0,0 +1,56 @@
from djmoney.contrib.django_rest_framework import MoneyField
from rest_framework import serializers
from core.models import Invoice
from core.component import component
class InvoiceComponentSerializer(serializers.ModelSerializer):
adjusted_end_date = serializers.DateTimeField()
price_charged = MoneyField(max_digits=10, decimal_places=0)
price_charged_currency = serializers.CharField(source="price_charged.currency")
def generate_invoice_component_serializer(model):
"""
Generate Invoice Component Serializer for particular model
:param model: The invoice component model
:return: serializer for particular model
"""
name = type(model).__name__
meta_params = {
"model": model,
"fields": "__all__"
}
meta_class = type("Meta", (object,), meta_params)
serializer_class = type(f"{name}Serializer", (InvoiceComponentSerializer,), {"Meta": meta_class})
return serializer_class
class InvoiceSerializer(serializers.ModelSerializer):
subtotal = MoneyField(max_digits=10, decimal_places=0)
subtotal_currency = serializers.CharField(source="subtotal.currency")
total = MoneyField(max_digits=10, decimal_places=0)
total_currency = serializers.CharField(source="total.currency", required=False)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
for field, model in component.INVOICE_COMPONENT_MODEL.items():
self.fields[field] = generate_invoice_component_serializer(model)(many=True)
class Meta:
model = Invoice
fields = '__all__'
class SimpleInvoiceSerializer(serializers.ModelSerializer):
subtotal = MoneyField(max_digits=10, decimal_places=0)
subtotal_currency = serializers.CharField(source="subtotal.currency")
total = MoneyField(max_digits=10, decimal_places=0)
total_currency = serializers.CharField(source="total.currency", required=False)
class Meta:
model = Invoice
fields = ['id', 'start_date', 'end_date', 'state', 'tax', 'subtotal', 'subtotal_currency', 'total',
'total_currency']

19
api/urls.py Normal file
View file

@ -0,0 +1,19 @@
from django.urls import path, include
from rest_framework import routers
from api import views
from core.component import component
router = routers.DefaultRouter()
for name, model in component.PRICE_MODEL.items():
router.register(f"price/{name}", views.get_generic_model_view_set(model))
router.register(r'settings', views.DynamicSettingViewSet, basename='settings')
router.register(r'invoice', views.InvoiceViewSet, basename='invoice')
router.register(r'admin_overview', views.AdminOverviewViewSet, basename='admin_overview')
router.register(r'project_overview', views.ProjectOverviewViewSet, basename='project_overview')
urlpatterns = [
path('', include(router.urls)),
path('api-auth/', include('rest_framework.urls')),
]

299
api/views.py Normal file
View file

@ -0,0 +1,299 @@
import dateutil.parser
import pytz
from django.db import transaction
from django.utils import timezone
from rest_framework import viewsets, serializers
from rest_framework.decorators import action
from rest_framework.response import Response
from api.serializers import InvoiceSerializer, SimpleInvoiceSerializer
from core.component import component
from core.component.component import INVOICE_COMPONENT_MODEL
from core.exception import PriceNotFound
from core.models import Invoice, BillingProject
from core.utils.dynamic_setting import get_dynamic_settings, get_dynamic_setting, set_dynamic_setting, BILLING_ENABLED
def get_generic_model_view_set(model):
name = type(model).__name__
meta_params = {
"model": model,
"fields": "__all__"
}
meta_class = type("Meta", (object,), meta_params)
serializer_class = type(f"{name}Serializer", (serializers.ModelSerializer,), {"Meta": meta_class})
view_set_params = {
"model": model,
"queryset": model.objects,
"serializer_class": serializer_class
}
return type(f"{name}ViewSet", (viewsets.ModelViewSet,), view_set_params)
class DynamicSettingViewSet(viewsets.ViewSet):
def list(self, request):
return Response(get_dynamic_settings())
def retrieve(self, request, pk=None):
return Response({
pk: get_dynamic_setting(pk)
})
def update(self, request, pk=None):
set_dynamic_setting(pk, request.data['value'])
return Response({
pk: get_dynamic_setting(pk)
})
def partial_update(self, request, pk=None):
set_dynamic_setting(pk, request.data['value'])
return Response({
pk: get_dynamic_setting(pk)
})
class InvoiceViewSet(viewsets.ModelViewSet):
serializer_class = InvoiceSerializer
def get_queryset(self):
tenant_id = self.request.query_params.get('tenant_id', None)
return Invoice.objects.filter(project__tenant_id=tenant_id).order_by('-start_date')
def parse_time(self, time):
dt = dateutil.parser.isoparse(time)
if not dt.tzinfo:
return pytz.UTC.localize(dt=dt)
return dt
@action(detail=False)
def simple_list(self, request):
serializer = SimpleInvoiceSerializer(self.get_queryset(), many=True)
return Response(serializer.data)
@action(detail=False, methods=['POST'])
def enable_billing(self, request):
try:
self.handle_init_billing(request.data)
return Response({
"status": "success"
})
except PriceNotFound as e:
return Response({
"message": str(e.identifier) + " price not found. Please check price configuration"
}, status=400)
@action(detail=False, methods=['POST'])
def disable_billing(self, request):
set_dynamic_setting(BILLING_ENABLED, False)
return Response({
"status": "success"
})
@action(detail=False, methods=['POST'])
def reset_billing(self, request):
self.handle_reset_billing()
return Response({
"status": "success"
})
@transaction.atomic
def handle_init_billing(self, data):
set_dynamic_setting(BILLING_ENABLED, True)
projects = {}
invoices = {}
date_today = timezone.now()
month_first_day = date_today.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
for name, handler in component.INVOICE_HANDLER.items():
payloads = data[name]
for payload in payloads:
if payload['tenant_id'] not in projects:
project, created = BillingProject.objects.get_or_create(tenant_id=payload['tenant_id'])
projects[payload['tenant_id']] = project
if payload['tenant_id'] not in invoices:
invoice = Invoice.objects.create(
project=projects[payload['tenant_id']],
start_date=month_first_day,
state=Invoice.InvoiceState.IN_PROGRESS
)
invoices[payload['tenant_id']] = invoice
start_date = self.parse_time(payload['start_date'])
if start_date < month_first_day:
start_date = month_first_day
payload['start_date'] = start_date
payload['invoice'] = invoices[payload['tenant_id']]
# create not accepting tenant_id, delete it
del payload['tenant_id']
handler.create(payload)
@transaction.atomic
def handle_reset_billing(self):
set_dynamic_setting(BILLING_ENABLED, False)
BillingProject.objects.all().delete()
for name, handler in component.INVOICE_HANDLER.items():
handler.delete()
for name, model in component.PRICE_MODEL.items():
model.objects.all().delete()
@action(detail=True)
def finish(self, request, pk):
invoice = Invoice.objects.filter(id=pk).first()
if invoice.state == Invoice.InvoiceState.UNPAID:
invoice.finish()
serializer = InvoiceSerializer(invoice)
return Response(serializer.data)
@action(detail=True)
def rollback_to_unpaid(self, request, pk):
invoice = Invoice.objects.filter(id=pk).first()
if invoice.state == Invoice.InvoiceState.FINISHED:
invoice.rollback_to_unpaid()
serializer = InvoiceSerializer(invoice)
return Response(serializer.data)
class AdminOverviewViewSet(viewsets.ViewSet):
def list(self, request):
return Response({})
@action(detail=False, methods=['GET'])
def total_resource(self, request):
data = {
'label': [],
'data': [],
}
for k, v in INVOICE_COMPONENT_MODEL.items():
data['label'].append(k)
data['data'].append(v.objects.filter(invoice__state=Invoice.InvoiceState.IN_PROGRESS).count())
return Response(data)
@action(detail=False, methods=['GET'])
def active_resource(self, request):
data = {
'label': [],
'data': [],
}
for k, v in INVOICE_COMPONENT_MODEL.items():
data['label'].append(k)
data['data'].append(
v.objects.filter(invoice__state=Invoice.InvoiceState.IN_PROGRESS, end_date=None).count())
return Response(data)
@action(detail=False, methods=['GET'])
def price_total_resource(self, request):
data = {
'label': [],
'data': [],
}
for k, v in INVOICE_COMPONENT_MODEL.items():
sum_of_price = sum([q.price_charged.amount for q in
v.objects.filter(invoice__state=Invoice.InvoiceState.IN_PROGRESS).all()])
data['label'].append(k)
data['data'].append(sum_of_price)
return Response(data)
@action(detail=False, methods=['GET'])
def price_active_resource(self, request):
data = {
'label': [],
'data': [],
}
for k, v in INVOICE_COMPONENT_MODEL.items():
sum_of_price = sum([q.price_charged.amount for q in
v.objects.filter(invoice__state=Invoice.InvoiceState.IN_PROGRESS, end_date=None).all()])
data['label'].append(k)
data['data'].append(sum_of_price)
return Response(data)
class ProjectOverviewViewSet(viewsets.ViewSet):
def list(self, request):
return Response({})
@action(detail=False, methods=['GET'])
def total_resource(self, request):
tenant_id = self.request.query_params.get('tenant_id', None)
project = BillingProject.objects.filter(tenant_id=tenant_id).first()
data = {
'label': [],
'data': [],
}
for k, v in INVOICE_COMPONENT_MODEL.items():
data['label'].append(k)
data['data'].append(
v.objects.filter(invoice__project=project, invoice__state=Invoice.InvoiceState.IN_PROGRESS).count())
return Response(data)
@action(detail=False, methods=['GET'])
def active_resource(self, request):
tenant_id = self.request.query_params.get('tenant_id', None)
project = BillingProject.objects.filter(tenant_id=tenant_id).first()
data = {
'label': [],
'data': [],
}
for k, v in INVOICE_COMPONENT_MODEL.items():
data['label'].append(k)
data['data'].append(
v.objects.filter(invoice__project=project, invoice__state=Invoice.InvoiceState.IN_PROGRESS,
end_date=None).count())
return Response(data)
@action(detail=False, methods=['GET'])
def price_total_resource(self, request):
tenant_id = self.request.query_params.get('tenant_id', None)
project = BillingProject.objects.filter(tenant_id=tenant_id).first()
data = {
'label': [],
'data': [],
}
for k, v in INVOICE_COMPONENT_MODEL.items():
sum_of_price = sum([q.price_charged.amount for q in
v.objects.filter(invoice__project=project,
invoice__state=Invoice.InvoiceState.IN_PROGRESS).all()])
data['label'].append(k)
data['data'].append(sum_of_price)
return Response(data)
@action(detail=False, methods=['GET'])
def price_active_resource(self, request):
tenant_id = self.request.query_params.get('tenant_id', None)
project = BillingProject.objects.filter(tenant_id=tenant_id).first()
data = {
'label': [],
'data': [],
}
for k, v in INVOICE_COMPONENT_MODEL.items():
sum_of_price = sum([q.price_charged.amount for q in
v.objects.filter(invoice__project=project,
invoice__state=Invoice.InvoiceState.IN_PROGRESS, end_date=None).all()])
data['label'].append(k)
data['data'].append(sum_of_price)
return Response(data)

6
bin/process_invoice.sh Executable file
View file

@ -0,0 +1,6 @@
#!/bin/bash
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
cd $SCRIPT_DIR || exit
cd ..
./env/bin/python manage.py process_invoice

16
bin/setup_api.sh Executable file
View file

@ -0,0 +1,16 @@
#!/bin/bash
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
cd $SCRIPT_DIR || exit
cd ..
echo "Installing Yuyu API Service"
yuyu_dir=`pwd -P`
echo "Yuyu dir is $yuyu_dir"
yuyu_dir_sub=${yuyu_dir//\//\\\/}
sed "s/{{yuyu_dir}}/$yuyu_dir_sub/g" "$yuyu_dir"/script/yuyu_api.service > /etc/systemd/system/yuyu_api.service
echo "Yuyu API Service Installed on /etc/systemd/system/yuyu_api.service"
echo "Done! you can enable Yuyu API with systemctl start yuyu_api"

17
bin/setup_event_monitor.sh Executable file
View file

@ -0,0 +1,17 @@
#!/bin/bash
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
cd $SCRIPT_DIR || exit
cd ..
echo "Installing Yuyu Event Monitor Service"
yuyu_dir=`pwd -P`
echo "Yuyu dir is $yuyu_dir"
yuyu_dir_sub=${yuyu_dir//\//\\\/}
sed "s/{{yuyu_dir}}/$yuyu_dir_sub/g" "$yuyu_dir"/script/yuyu_event_monitor.service > /etc/systemd/system/yuyu_event_monitor.service
echo "Yuyu Event Monitor Service Installed on /etc/systemd/system/yuyu_event_monitor.service"
echo "Done! you can enable Yuyu Event Monitor with systemctl start yuyu_event_monitor"

0
core/__init__.py Normal file
View file

79
core/admin.py Normal file
View file

@ -0,0 +1,79 @@
from django.contrib import admin
from core.models import FlavorPrice, VolumePrice, FloatingIpsPrice, BillingProject, Invoice, InvoiceVolume, \
InvoiceFloatingIp, InvoiceInstance, DynamicSetting, InvoiceImage, ImagePrice, SnapshotPrice, RouterPrice, \
InvoiceSnapshot, InvoiceRouter
@admin.register(DynamicSetting)
class FlavorPriceAdmin(admin.ModelAdmin):
list_display = ('key', 'value', 'type')
@admin.register(FlavorPrice)
class FlavorPriceAdmin(admin.ModelAdmin):
list_display = ('flavor_id', 'hourly_price', 'monthly_price')
@admin.register(FloatingIpsPrice)
class FloatingIpsPriceAdmin(admin.ModelAdmin):
list_display = ('hourly_price', 'monthly_price')
@admin.register(VolumePrice)
class VolumePriceAdmin(admin.ModelAdmin):
list_display = ('volume_type_id', 'hourly_price', 'monthly_price')
@admin.register(RouterPrice)
class RouterPriceAdmin(admin.ModelAdmin):
list_display = ('hourly_price', 'monthly_price')
@admin.register(SnapshotPrice)
class SnapshotPriceAdmin(admin.ModelAdmin):
list_display = ('hourly_price', 'monthly_price')
@admin.register(ImagePrice)
class ImagePriceAdmin(admin.ModelAdmin):
list_display = ('hourly_price', 'monthly_price')
@admin.register(BillingProject)
class BillingProjectAdmin(admin.ModelAdmin):
list_display = ('tenant_id',)
@admin.register(Invoice)
class InvoiceAdmin(admin.ModelAdmin):
list_display = ('__str__', 'project', 'start_date')
@admin.register(InvoiceInstance)
class InvoiceInstanceAdmin(admin.ModelAdmin):
list_display = ('instance_id',)
@admin.register(InvoiceFloatingIp)
class InvoiceFloatingIpAdmin(admin.ModelAdmin):
list_display = ('fip_id',)
@admin.register(InvoiceVolume)
class InvoiceVolumeAdmin(admin.ModelAdmin):
list_display = ('volume_id',)
@admin.register(InvoiceRouter)
class InvoiceRouterAdmin(admin.ModelAdmin):
list_display = ('router_id', 'name')
@admin.register(InvoiceSnapshot)
class InvoiceSnapshotAdmin(admin.ModelAdmin):
list_display = ('snapshot_id', 'name', 'space_allocation_gb')
@admin.register(InvoiceImage)
class InvoiceImageAdmin(admin.ModelAdmin):
list_display = ('image_id', 'name', 'space_allocation_gb')

6
core/apps.py Normal file
View file

@ -0,0 +1,6 @@
from django.apps import AppConfig
class CoreConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'core'

View file

View file

@ -0,0 +1,123 @@
import abc
from django.db import transaction
from django.utils import timezone
from core.models import Invoice, BillingProject
from core.component.base.invoice_handler import InvoiceHandler
class EventHandler(metaclass=abc.ABCMeta):
def __init__(self, invoice_handler):
self.invoice_handler: InvoiceHandler = invoice_handler
def get_tenant_progress_invoice(self, tenant_id):
"""
Get in progress invoice for specific tenant id.
Will create new instance if active invoice not found.
And will create new billing project if tenant id not found.
:param tenant_id: Tenant id to get the invoice from.
:return:
"""
invoice = Invoice.objects.filter(project__tenant_id=tenant_id, state=Invoice.InvoiceState.IN_PROGRESS).first()
if not invoice:
project = BillingProject.objects.get_or_create(tenant_id=tenant_id)
date_today = timezone.now()
month_first_day = date_today.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
invoice = Invoice.objects.create(
project=project,
start_date=month_first_day,
state=Invoice.InvoiceState.IN_PROGRESS
)
return invoice
@abc.abstractmethod
def handle(self, event_type, raw_payload):
"""
Handle event from the message queue
:param event_type: The event type
:param raw_payload: Payload inside the message
:return:
"""
raise NotImplementedError()
@abc.abstractmethod
def clean_payload(self, event_type, raw_payload):
"""
Clean raw payload into payload that can be accepted by invoice handler
:param event_type: Current event type
:param raw_payload: Raw payload from messaging queue
:return:
"""
raise NotImplementedError()
@transaction.atomic
def handle_create(self, invoice: Invoice, event_type, raw_payload):
"""
Create new invoice component that will be saved to current invoice.
You need to call this method manually from handle() if you want to use it.
:param invoice: The invoice that will be saved into
:param event_type: Current event type
:param raw_payload: Raw payload from messaging queue.
:return:
"""
payload = self.clean_payload(event_type, raw_payload)
instance = self.invoice_handler.get_active_instance(invoice, payload)
if not instance:
payload['invoice'] = invoice
payload['start_date'] = timezone.now()
self.invoice_handler.create(payload, fallback_price=True)
return True
return False
@transaction.atomic
def handle_delete(self, invoice: Invoice, event_type, raw_payload):
"""
Close invoice component when delete event occurred.
You need to call this method manually from handle() if you want to use it.
:param invoice: The invoice that will be saved into
:param event_type: Current event type
:param raw_payload: Raw payload from messaging queue.
:return:
"""
payload = self.clean_payload(event_type, raw_payload)
instance = self.invoice_handler.get_active_instance(invoice, payload)
if instance:
self.invoice_handler.update_and_close(instance, payload)
return True
return False
@transaction.atomic
def handle_update(self, invoice: Invoice, event_type, raw_payload):
"""
Update invoice component when update event occurred.
You need to call this method manually from handle() if you want to use it.
:param invoice: The invoice that will be saved into
:param event_type: Current event type
:param raw_payload: Raw payload from messaging queue.
:return:
"""
payload = self.clean_payload(event_type, raw_payload)
instance = self.invoice_handler.get_active_instance(invoice, payload)
if instance:
if self.invoice_handler.is_price_dependency_changed(instance, payload):
self.invoice_handler.roll(instance, close_date=timezone.now(), update_payload=payload, fallback_price=True)
return True
if self.invoice_handler.is_informative_changed(instance, payload):
self.invoice_handler.update(instance, update_payload=payload)
return True
return False

View file

@ -0,0 +1,167 @@
import abc
from django.conf import settings
from django.utils import timezone
from djmoney.money import Money
from core.exception import PriceNotFound
from core.models import InvoiceComponentMixin, PriceMixin
class InvoiceHandler(metaclass=abc.ABCMeta):
INVOICE_CLASS = None
KEY_FIELD = None
INFORMATIVE_FIELDS = []
PRICE_DEPENDENCY_FIELDS = []
def create(self, payload, fallback_price=False):
"""
Create new invoice component
:param payload: the data that will be created
:param fallback_price: Whether use 0 price if price not found
:return:
"""
try:
price = self.get_price(payload)
if price is None:
raise PriceNotFound()
except PriceNotFound:
if fallback_price:
price = PriceMixin()
price.hourly_price = Money(amount=0, currency=settings.DEFAULT_CURRENCY)
price.monthly_price = Money(amount=0, currency=settings.DEFAULT_CURRENCY)
else:
raise
payload['hourly_price'] = price.hourly_price
payload['monthly_price'] = price.monthly_price
self.INVOICE_CLASS.objects.create(**payload)
def delete(self):
self.INVOICE_CLASS.objects.all().delete()
def roll(self, instance: InvoiceComponentMixin, close_date, update_payload=None, fallback_price=False):
"""
Roll current instance.
Close current component instance and clone it
:param instance: The instance that want to be rolled
:param close_date: The close date of current instance
:param update_payload: New data to update the next component instance
:param fallback_price: Whether use 0 price if price not found
:return:
"""
if update_payload is None:
update_payload = {}
if not instance.is_closed():
instance.close(close_date)
# Set primary ke to None, this will make save() to create a new row
instance.pk = None
instance.start_date = instance.end_date
instance.end_date = None
instance.created_at = None
instance.updated_at = None
# Update the new instance without saving
instance = self.update(instance, update_payload, save=False)
# Update the price
try:
price = self.get_price(self.get_price_dependency_from_instance(instance))
if price is None:
raise PriceNotFound()
except PriceNotFound:
if fallback_price:
price = PriceMixin()
price.hourly_price = Money(amount=0, currency=settings.DEFAULT_CURRENCY)
price.monthly_price = Money(amount=0, currency=settings.DEFAULT_CURRENCY)
else:
raise
instance.hourly_price = price.hourly_price
instance.monthly_price = price.monthly_price
instance.save()
return instance
def update(self, instance, update_payload, save=True):
"""
Update instance
:param instance: instance that will be updated
:param update_payload: new data
:param save: will it be saved or not
:return:
"""
for key, value in update_payload.items():
setattr(instance, key, value)
if save:
instance.save()
return instance
def update_and_close(self, instance, payload):
"""
:param instance: Instance that will be closed
:param payload: update payload
:return:
"""
self.update(instance, payload, save=False)
instance.close(timezone.now()) # Close will also save the instance
def is_informative_changed(self, instance, payload):
"""
Check whether informative field in instance is changed compared to the payload
:param instance: the instance that will be checked
:param payload: payload to compare
:return:
"""
for informative in self.INFORMATIVE_FIELDS:
if getattr(instance, informative) != payload[informative]:
return True
return False
def is_price_dependency_changed(self, instance, payload):
"""
Check whether price dependency field in instance is changed compared to the payload
:param instance: the instance that will be checked
:param payload: payload to compare
:return:
"""
for price_dependency in self.PRICE_DEPENDENCY_FIELDS:
if getattr(instance, price_dependency) != payload[price_dependency]:
return True
return False
def get_active_instance(self, invoice, payload):
"""
Get currently active invoice component instance.
Filtered by invoice and key field in payload
:param invoice: Invoice target
:param payload: Payload to get key field, please make sure there are value for the key field inside the payload
:return:
"""
kwargs = {"invoice": invoice, "end_date": None, self.KEY_FIELD: payload[self.KEY_FIELD]}
return self.INVOICE_CLASS.objects.filter(**kwargs).first()
def get_price_dependency_from_instance(self, instance):
"""
Get payload with price dependency field extracted from instance
:param instance: Instance to extract the price dependency
:return:
"""
return {field: getattr(instance, field) for field in self.PRICE_DEPENDENCY_FIELDS}
@abc.abstractmethod
def get_price(self, payload) -> PriceMixin:
"""
Get price based on payload
:param payload: the payload that will contain filter to get the price
:return:
"""
raise NotImplementedError()

View file

@ -0,0 +1,65 @@
from core.component.image.event_handler import ImageEventHandler
from core.component.image.invoice_handler import ImageInvoiceHandler
from core.component.router.event_handler import RouterEventHandler
from core.component.router.invoice_handler import RouterInvoiceHandler
from core.component.snapshot.event_handler import SnapshotEventHandler
from core.component.snapshot.invoice_handler import SnapshotInvoiceHandler
from core.models import InvoiceInstance, InvoiceVolume, InvoiceFloatingIp, FlavorPrice, FloatingIpsPrice, VolumePrice, \
RouterPrice, SnapshotPrice, InvoiceRouter, InvoiceSnapshot, ImagePrice, InvoiceImage
from core.component.floating_ips.event_handler import FloatingIpEventHandler
from core.component.floating_ips.invoice_handler import FloatingIpInvoiceHandler
from core.component.instances.event_handler import InstanceEventHandler
from core.component.instances.invoice_handler import InstanceInvoiceHandler
from core.component.labels import LABEL_INSTANCES, LABEL_VOLUMES, LABEL_FLOATING_IPS, LABEL_ROUTERS, LABEL_SNAPSHOTS, \
LABEL_IMAGES
from core.component.volume.event_handler import VolumeEventHandler
from core.component.volume.invoice_handler import VolumeInvoiceHandler
"""
Define a model that represent price for particular component
"""
PRICE_MODEL = {
"flavor": FlavorPrice,
"floating_ip": FloatingIpsPrice,
"volume": VolumePrice,
"router": RouterPrice,
"snapshot": SnapshotPrice,
"image": ImagePrice
}
"""
Define a model that represent a component in invoice.
The label that used for the key must be able to access through [Invoice] model
"""
INVOICE_COMPONENT_MODEL = {
LABEL_INSTANCES: InvoiceInstance,
LABEL_VOLUMES: InvoiceVolume,
LABEL_FLOATING_IPS: InvoiceFloatingIp,
LABEL_ROUTERS: InvoiceRouter,
LABEL_SNAPSHOTS: InvoiceSnapshot,
LABEL_IMAGES: InvoiceImage
}
"""
Define a class that handle the event from message queue
"""
EVENT_HANDLER = {
LABEL_INSTANCES: InstanceEventHandler,
LABEL_VOLUMES: VolumeEventHandler,
LABEL_FLOATING_IPS: FloatingIpEventHandler,
LABEL_ROUTERS: RouterEventHandler,
LABEL_SNAPSHOTS: SnapshotEventHandler,
LABEL_IMAGES: ImageEventHandler
}
"""
Define an instance that handle invoice creation or update
"""
INVOICE_HANDLER = {
LABEL_INSTANCES: InstanceInvoiceHandler(),
LABEL_VOLUMES: VolumeInvoiceHandler(),
LABEL_FLOATING_IPS: FloatingIpInvoiceHandler(),
LABEL_ROUTERS: RouterInvoiceHandler(),
LABEL_SNAPSHOTS: SnapshotInvoiceHandler(),
LABEL_IMAGES: ImageInvoiceHandler()
}

View file

@ -0,0 +1,23 @@
from core.component.base.event_handler import EventHandler
from core.component.floating_ips.invoice_handler import FloatingIpInvoiceHandler
class FloatingIpEventHandler(EventHandler):
def handle(self, event_type, raw_payload):
if event_type == 'floatingip.create.end':
tenant_id = raw_payload['floatingip']['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_create(invoice, event_type, raw_payload)
if event_type == 'floatingip.delete.end':
tenant_id = raw_payload['floatingip']['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_delete(invoice, event_type, raw_payload)
def clean_payload(self, event_type, raw_payload):
payload = {
"fip_id": raw_payload['floatingip']['id'],
"ip": raw_payload['floatingip']['floating_ip_address'],
}
return payload

View file

@ -0,0 +1,19 @@
from core.exception import PriceNotFound
from core.models import FloatingIpsPrice, InvoiceFloatingIp, PriceMixin
from core.component.base.invoice_handler import InvoiceHandler
class FloatingIpInvoiceHandler(InvoiceHandler):
INVOICE_CLASS = InvoiceFloatingIp
KEY_FIELD = "fip_id"
PRICE_DEPENDENCY_FIELDS = []
INFORMATIVE_FIELDS = ["ip"]
def get_price(self, payload) -> PriceMixin:
price = FloatingIpsPrice.objects.first()
if price is None:
raise PriceNotFound(identifier='floating ip')
return price

View file

@ -0,0 +1,28 @@
from core.component.base.event_handler import EventHandler
class ImageEventHandler(EventHandler):
def handle(self, event_type, raw_payload):
if event_type == 'image.activate':
tenant_id = raw_payload['owner']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_create(invoice, event_type, raw_payload)
if event_type == 'image.delete':
tenant_id = raw_payload['owner']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_delete(invoice, event_type, raw_payload)
if event_type == 'image.update':
tenant_id = raw_payload['owner']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_update(invoice, event_type, raw_payload)
def clean_payload(self, event_type, raw_payload):
payload = {
"image_id": raw_payload['id'],
"space_allocation_gb": raw_payload['size'] / 1024 / 1024 / 1024,
"name": raw_payload['name'] or raw_payload['id'],
}
return payload

View file

@ -0,0 +1,17 @@
from core.component.base.invoice_handler import InvoiceHandler
from core.exception import PriceNotFound
from core.models import PriceMixin, InvoiceImage, ImagePrice
class ImageInvoiceHandler(InvoiceHandler):
INVOICE_CLASS = InvoiceImage
KEY_FIELD = "image_id"
PRICE_DEPENDENCY_FIELDS = ["space_allocation_gb"]
INFORMATIVE_FIELDS = ["name"]
def get_price(self, payload) -> PriceMixin:
price = ImagePrice.objects.first()
if price is None:
raise PriceNotFound(identifier='image')
return price

View file

@ -0,0 +1,25 @@
from core.component.base.event_handler import EventHandler
class InstanceEventHandler(EventHandler):
def handle(self, event_type, raw_payload):
if event_type == 'compute.instance.update':
tenant_id = raw_payload['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
is_updated = self.handle_update(invoice, event_type, raw_payload)
if not is_updated and raw_payload['state'] == 'active':
self.handle_create(invoice, event_type, raw_payload)
if not is_updated and raw_payload['state'] == 'deleted':
self.handle_delete(invoice, event_type, raw_payload)
def clean_payload(self, event_type, raw_payload):
payload = {
"instance_id": raw_payload['instance_id'],
"flavor_id": raw_payload['instance_flavor_id'],
"name": raw_payload['display_name'],
}
return payload

View file

@ -0,0 +1,18 @@
from core.exception import PriceNotFound
from core.models import FlavorPrice, InvoiceInstance, PriceMixin
from core.component.base.invoice_handler import InvoiceHandler
class InstanceInvoiceHandler(InvoiceHandler):
INVOICE_CLASS = InvoiceInstance
KEY_FIELD = "instance_id"
PRICE_DEPENDENCY_FIELDS = ['flavor_id']
INFORMATIVE_FIELDS = ['name']
def get_price(self, payload) -> PriceMixin:
price = FlavorPrice.objects.filter(flavor_id=payload['flavor_id']).first()
if price is None:
raise PriceNotFound(identifier='flavor')
return price

9
core/component/labels.py Normal file
View file

@ -0,0 +1,9 @@
LABEL_INSTANCES = "instances"
LABEL_VOLUMES = "volumes"
LABEL_FLOATING_IPS = "floating_ips"
LABEL_ROUTERS = "routers"
LABEL_SNAPSHOTS = "snapshots"
LABEL_IMAGES = "images"
INVOICE_COMPONENT_LABELS = [LABEL_INSTANCES, LABEL_VOLUMES, LABEL_FLOATING_IPS, LABEL_ROUTERS, LABEL_SNAPSHOTS,
LABEL_IMAGES]

View file

@ -0,0 +1,43 @@
from core.component.base.event_handler import EventHandler
from core.component.floating_ips.invoice_handler import FloatingIpInvoiceHandler
class RouterEventHandler(EventHandler):
def is_external_gateway_set(self, raw_payload):
return raw_payload['router']['external_gateway_info'] is not None
def handle(self, event_type, raw_payload):
# Case: Creating router with external gateway
if event_type == 'router.create.end' and self.is_external_gateway_set(raw_payload):
tenant_id = raw_payload['router']['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_create(invoice, event_type, raw_payload)
if event_type == 'router.update.end':
tenant_id = raw_payload['router']['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
# Handel update for existing instance
self.handle_update(invoice, event_type, raw_payload)
# Case: Existing router set gateway
if self.is_external_gateway_set(raw_payload):
self.handle_create(invoice, event_type, raw_payload)
# Case: Existing router remove gateway
if not self.is_external_gateway_set(raw_payload):
self.handle_delete(invoice, event_type, raw_payload)
# Case: Delete router
if event_type == 'router.delete.end':
tenant_id = raw_payload['router']['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_delete(invoice, event_type, raw_payload)
def clean_payload(self, event_type, raw_payload):
payload = {
"router_id": raw_payload['router']['id'],
"name": raw_payload['router']['name'],
}
return payload

View file

@ -0,0 +1,17 @@
from core.component.base.invoice_handler import InvoiceHandler
from core.exception import PriceNotFound
from core.models import PriceMixin, InvoiceRouter, RouterPrice
class RouterInvoiceHandler(InvoiceHandler):
INVOICE_CLASS = InvoiceRouter
KEY_FIELD = "router_id"
PRICE_DEPENDENCY_FIELDS = []
INFORMATIVE_FIELDS = ["name"]
def get_price(self, payload) -> PriceMixin:
price = RouterPrice.objects.first()
if price is None:
raise PriceNotFound(identifier='router')
return price

View file

@ -0,0 +1,29 @@
from core.component.base.event_handler import EventHandler
from core.component.floating_ips.invoice_handler import FloatingIpInvoiceHandler
class SnapshotEventHandler(EventHandler):
def handle(self, event_type, raw_payload):
if event_type == 'snapshot.create.end':
tenant_id = raw_payload['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_create(invoice, event_type, raw_payload)
if event_type == 'snapshot.delete.end':
tenant_id = raw_payload['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_delete(invoice, event_type, raw_payload)
if event_type == 'snapshot.update.end':
tenant_id = raw_payload['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_update(invoice, event_type, raw_payload)
def clean_payload(self, event_type, raw_payload):
payload = {
"snapshot_id": raw_payload['snapshot_id'],
"space_allocation_gb": raw_payload['volume_size'],
"name": raw_payload['display_name'],
}
return payload

View file

@ -0,0 +1,17 @@
from core.component.base.invoice_handler import InvoiceHandler
from core.exception import PriceNotFound
from core.models import PriceMixin, InvoiceSnapshot, SnapshotPrice
class SnapshotInvoiceHandler(InvoiceHandler):
INVOICE_CLASS = InvoiceSnapshot
KEY_FIELD = "snapshot_id"
PRICE_DEPENDENCY_FIELDS = ["space_allocation_gb"]
INFORMATIVE_FIELDS = ["name"]
def get_price(self, payload) -> PriceMixin:
price = SnapshotPrice.objects.first()
if price is None:
raise PriceNotFound(identifier='snapshot')
return price

View file

@ -0,0 +1,29 @@
from core.component.base.event_handler import EventHandler
class VolumeEventHandler(EventHandler):
def handle(self, event_type, raw_payload):
if event_type == 'volume.create.end':
tenant_id = raw_payload['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_create(invoice, event_type, raw_payload)
if event_type == 'volume.delete.end':
tenant_id = raw_payload['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_delete(invoice, event_type, raw_payload)
if event_type in ['volume.resize.end', 'volume.update.end', 'volume.retype']:
tenant_id = raw_payload['tenant_id']
invoice = self.get_tenant_progress_invoice(tenant_id)
self.handle_update(invoice, event_type, raw_payload)
def clean_payload(self, event_type, raw_payload):
payload = {
"volume_id": raw_payload['volume_id'],
"volume_type_id": raw_payload['volume_type'],
"volume_name": raw_payload['display_name'] or raw_payload['volume_id'],
"space_allocation_gb": raw_payload['size'],
}
return payload

View file

@ -0,0 +1,18 @@
from core.exception import PriceNotFound
from core.models import VolumePrice, InvoiceVolume, PriceMixin
from core.component.base.invoice_handler import InvoiceHandler
class VolumeInvoiceHandler(InvoiceHandler):
INVOICE_CLASS = InvoiceVolume
KEY_FIELD = "volume_id"
PRICE_DEPENDENCY_FIELDS = ['volume_type_id', 'space_allocation_gb']
INFORMATIVE_FIELDS = ['volume_name']
def get_price(self, payload) -> PriceMixin:
price = VolumePrice.objects.filter(volume_type_id=payload['volume_type_id']).first()
if price is None:
raise PriceNotFound(identifier='volume')
return price

29
core/event_endpoint.py Normal file
View file

@ -0,0 +1,29 @@
import logging
from oslo_messaging import NotificationResult
from core.component import component
from core.utils.dynamic_setting import get_dynamic_settings, BILLING_ENABLED, get_dynamic_setting
LOG = logging.getLogger("yuyu_notification")
class EventEndpoint(object):
def __init__(self):
self.event_handler = [
cls(component.INVOICE_HANDLER[label]) for label, cls in component.EVENT_HANDLER.items()
]
def info(self, ctxt, publisher_id, event_type, payload, metadata):
LOG.info("=== Event Received ===")
LOG.info("Event Type: " + str(event_type))
LOG.info("Payload: " + str(payload))
if not get_dynamic_setting(BILLING_ENABLED):
return NotificationResult.HANDLED
# TODO: Error Handling
for handler in self.event_handler:
handler.handle(event_type, payload)
return NotificationResult.HANDLED

3
core/exception.py Normal file
View file

@ -0,0 +1,3 @@
class PriceNotFound(Exception):
def __init__(self, identifier=None):
self.identifier = identifier

View file

View file

View file

@ -0,0 +1,71 @@
from django.conf import settings
import logging
import os
import signal
import time
from oslo_config import cfg
import oslo_messaging as messaging
from oslo_messaging import notify # noqa
from django.core.management.base import BaseCommand
from core.event_endpoint import EventEndpoint
LOG = logging.getLogger("yuyu_notification")
class SignalExit(SystemExit):
def __init__(self, signo, exccode=1):
super(SignalExit, self).__init__(exccode)
self.signo = signo
class Command(BaseCommand):
help = 'Start Yuyu Event Monitor'
def signal_handler(self, signum, frame):
raise SignalExit(signum)
def run_server(self, transport, server):
try:
server.start()
server.wait()
LOG.info('The server is terminating')
time.sleep(1)
except SignalExit as e:
LOG.info('Signal %s is caught. Interrupting the execution',
e.signo)
server.stop()
server.wait()
finally:
transport.cleanup()
def notify_server(self, transport, topics):
endpoints = [EventEndpoint()]
targets = list(map(lambda t: messaging.Target(topic=t), topics))
server = notify.get_notification_listener(
transport,
targets,
endpoints,
executor='threading'
)
self.run_server(transport, server)
def handle(self, *args, **options):
url = settings.YUYU_NOTIFICATION_URL
transport = messaging.get_notification_transport(cfg.CONF,
url=url)
# oslo.config defaults
cfg.CONF.heartbeat_interval = 5
cfg.CONF.prog = os.path.basename(__file__)
cfg.CONF.project = 'yuyu'
signal.signal(signal.SIGTERM, self.signal_handler)
signal.signal(signal.SIGINT, self.signal_handler)
self.notify_server(
transport=transport,
topics=settings.YUYU_NOTIFICATION_TOPICS,
)

View file

@ -0,0 +1,57 @@
import logging
from typing import Mapping, Dict, Iterable
from django.core.management import BaseCommand
from django.utils import timezone
from core.models import Invoice, InvoiceComponentMixin
from core.component import component, labels
from core.utils.dynamic_setting import get_dynamic_setting, BILLING_ENABLED, INVOICE_TAX
LOG = logging.getLogger("yuyu_new_invoice")
class Command(BaseCommand):
help = 'Yuyu New Invoice'
def handle(self, *args, **options):
print("Processing Invoice")
if not get_dynamic_setting(BILLING_ENABLED):
return
self.close_date = timezone.now()
self.tax_pertentage = get_dynamic_setting(INVOICE_TAX)
active_invoices = Invoice.objects.filter(state=Invoice.InvoiceState.IN_PROGRESS).all()
for active_invoice in active_invoices:
self.close_active_invoice(active_invoice)
print("Processing Done")
def close_active_invoice(self, active_invoice: Invoice):
active_components_map: Dict[str, Iterable[InvoiceComponentMixin]] = {}
for label in labels.INVOICE_COMPONENT_LABELS:
active_components_map[label] = getattr(active_invoice, label).filter(end_date=None).all()
# Close Invoice Component
for active_component in active_components_map[label]:
active_component.close(self.close_date)
# Finish current invoice
active_invoice.close(self.close_date, self.tax_pertentage)
# Creating new Invoice
new_invoice = Invoice.objects.create(
project=active_invoice.project,
start_date=self.close_date,
state=Invoice.InvoiceState.IN_PROGRESS
)
new_invoice.save()
# Cloning active component to continue in next invoice
for label, active_components in active_components_map.items():
handler = component.INVOICE_HANDLER[label]
for active_component in active_components:
handler.roll(active_component, self.close_date, update_payload={
"invoice": new_invoice
}, fallback_price=True)

View file

@ -0,0 +1,157 @@
# Generated by Django 3.2.6 on 2021-10-10 13:08
from django.db import migrations, models
import django.db.models.deletion
import djmoney.models.fields
class Migration(migrations.Migration):
initial = True
dependencies = [
]
operations = [
migrations.CreateModel(
name='BillingProject',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('tenant_id', models.CharField(max_length=256)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='FlavorPrice',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('flavor_id', models.CharField(max_length=256)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='FloatingIpsPrice',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='Invoice',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('start_date', models.DateTimeField()),
('end_date', models.DateTimeField(blank=True, default=None, null=True)),
('state', models.IntegerField(choices=[(1, 'In Progress'), (100, 'Finished')])),
('tax_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('tax', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('total_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('total', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('project', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='core.billingproject')),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='VolumePrice',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('volume_type_id', models.CharField(max_length=256)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='InvoiceVolume',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('start_date', models.DateTimeField()),
('end_date', models.DateTimeField(blank=True, default=None, null=True)),
('volume_id', models.CharField(max_length=256)),
('volume_type_id', models.CharField(max_length=256)),
('space_allocation_gb', models.IntegerField()),
('volume_name', models.CharField(max_length=256)),
('invoice', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='volumes', to='core.invoice')),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='InvoiceInstance',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('start_date', models.DateTimeField()),
('end_date', models.DateTimeField(blank=True, default=None, null=True)),
('instance_id', models.CharField(max_length=266)),
('flavor_id', models.CharField(max_length=256)),
('name', models.CharField(max_length=256)),
('invoice', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='instances', to='core.invoice')),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='InvoiceFloatingIp',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('start_date', models.DateTimeField()),
('end_date', models.DateTimeField(blank=True, default=None, null=True)),
('fip_id', models.CharField(max_length=266)),
('ip', models.CharField(max_length=256)),
('invoice', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='floating_ips', to='core.invoice')),
],
options={
'abstract': False,
},
),
]

View file

@ -0,0 +1,25 @@
# Generated by Django 3.2.6 on 2021-10-13 16:38
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0001_initial'),
]
operations = [
migrations.CreateModel(
name='DynamicSetting',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('key', models.CharField(db_index=True, max_length=256, unique=True)),
('value', models.TextField()),
('type', models.IntegerField(choices=[(1, 'Boolean'), (2, 'Int'), (3, 'Str'), (4, 'Json')])),
],
options={
'abstract': False,
},
),
]

View file

@ -0,0 +1,18 @@
# Generated by Django 3.2.6 on 2021-10-13 17:55
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0002_dynamicsetting'),
]
operations = [
migrations.AlterField(
model_name='invoicevolume',
name='space_allocation_gb',
field=models.FloatField(),
),
]

View file

@ -0,0 +1,86 @@
# Generated by Django 3.2.6 on 2021-10-14 05:47
from django.db import migrations, models
import django.db.models.deletion
import djmoney.models.fields
class Migration(migrations.Migration):
dependencies = [
('core', '0003_alter_invoicevolume_space_allocation_gb'),
]
operations = [
migrations.CreateModel(
name='RouterPrice',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='SnapshotPrice',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='InvoiceSnapshot',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('start_date', models.DateTimeField()),
('end_date', models.DateTimeField(blank=True, default=None, null=True)),
('snapshot_id', models.CharField(max_length=256)),
('space_allocation_gb', models.FloatField()),
('name', models.CharField(max_length=256)),
('invoice', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='snapshots', to='core.invoice')),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='InvoiceRouter',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('start_date', models.DateTimeField()),
('end_date', models.DateTimeField(blank=True, default=None, null=True)),
('router_id', models.CharField(max_length=256)),
('name', models.CharField(max_length=256)),
('invoice', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='routers', to='core.invoice')),
],
options={
'abstract': False,
},
),
]

View file

@ -0,0 +1,51 @@
# Generated by Django 3.2.6 on 2021-10-29 04:22
from django.db import migrations, models
import django.db.models.deletion
import djmoney.models.fields
class Migration(migrations.Migration):
dependencies = [
('core', '0004_invoicerouter_invoicesnapshot_routerprice_snapshotprice'),
]
operations = [
migrations.CreateModel(
name='ImagePrice',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
],
options={
'abstract': False,
},
),
migrations.CreateModel(
name='InvoiceImage',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True)),
('updated_at', models.DateTimeField(auto_now=True)),
('hourly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('hourly_price', djmoney.models.fields.MoneyField(decimal_places=0, max_digits=10)),
('monthly_price_currency', djmoney.models.fields.CurrencyField(choices=[('IDR', 'Indonesian Rupiah'), ('USD', 'US Dollar')], default='IDR', editable=False, max_length=3)),
('monthly_price', djmoney.models.fields.MoneyField(blank=True, decimal_places=0, default=None, max_digits=10, null=True)),
('start_date', models.DateTimeField()),
('end_date', models.DateTimeField(blank=True, default=None, null=True)),
('image_id', models.CharField(max_length=256)),
('space_allocation_gb', models.FloatField()),
('name', models.CharField(max_length=256)),
('invoice', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='images', to='core.invoice')),
],
options={
'abstract': False,
},
),
]

View file

@ -0,0 +1,18 @@
# Generated by Django 3.2.6 on 2022-02-13 18:51
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0005_imageprice_invoiceimage'),
]
operations = [
migrations.AlterField(
model_name='invoice',
name='state',
field=models.IntegerField(choices=[(1, 'In Progress'), (2, 'Unpaid'), (100, 'Finished')]),
),
]

View file

@ -0,0 +1,18 @@
# Generated by Django 3.2.6 on 2022-02-13 18:59
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0006_alter_invoice_state'),
]
operations = [
migrations.AddField(
model_name='invoice',
name='finish_date',
field=models.DateTimeField(blank=True, default=None, null=True),
),
]

View file

206
core/models.py Normal file
View file

@ -0,0 +1,206 @@
import math
from django.conf import settings
from django.db import models
from django.utils import timezone
from djmoney.models.fields import MoneyField
from djmoney.money import Money
from core.component import labels
from core.utils.model_utils import BaseModel, TimestampMixin, PriceMixin, InvoiceComponentMixin
# region Dynamic Setting
class DynamicSetting(BaseModel):
class DataType(models.IntegerChoices):
BOOLEAN = 1
INT = 2
STR = 3
JSON = 4
key = models.CharField(max_length=256, unique=True, db_index=True)
value = models.TextField()
type = models.IntegerField(choices=DataType.choices)
# end region
# region Pricing
class FlavorPrice(BaseModel, TimestampMixin, PriceMixin):
flavor_id = models.CharField(max_length=256)
class FloatingIpsPrice(BaseModel, TimestampMixin, PriceMixin):
# No need for any additional field
pass
class VolumePrice(BaseModel, TimestampMixin, PriceMixin):
volume_type_id = models.CharField(max_length=256)
class RouterPrice(BaseModel, TimestampMixin, PriceMixin):
# No need for any additional field
pass
class SnapshotPrice(BaseModel, TimestampMixin, PriceMixin):
# No need for any additional field
pass
class ImagePrice(BaseModel, TimestampMixin, PriceMixin):
# No need for any additional field
pass
# end region
# region Invoicing
class BillingProject(BaseModel, TimestampMixin):
tenant_id = models.CharField(max_length=256)
def __str__(self):
return self.tenant_id
class Invoice(BaseModel, TimestampMixin):
class InvoiceState(models.IntegerChoices):
IN_PROGRESS = 1
UNPAID = 2
FINISHED = 100
project = models.ForeignKey('BillingProject', on_delete=models.CASCADE)
start_date = models.DateTimeField()
end_date = models.DateTimeField(default=None, blank=True, null=True)
finish_date = models.DateTimeField(default=None, blank=True, null=True)
state = models.IntegerField(choices=InvoiceState.choices)
tax = MoneyField(max_digits=10, decimal_places=0, default=None, blank=True, null=True)
total = MoneyField(max_digits=10, decimal_places=0, default=None, blank=True, null=True)
@property
def subtotal(self):
# Need to optimize?
# currently price is calculated on the fly, maybe need to save to db to make performance faster?
# or using cache?
price = 0
for component_relation_label in labels.INVOICE_COMPONENT_LABELS:
relation_all_row = getattr(self, component_relation_label).all()
price += sum(map(lambda x: x.price_charged, relation_all_row))
if price == 0:
return Money(amount=price, currency=settings.DEFAULT_CURRENCY)
return price
@property
def total_resource(self):
total = 0
for component_relation_label in labels.INVOICE_COMPONENT_LABELS:
total += getattr(self, component_relation_label).count()
return total
def close(self, date, tax_percentage):
self.state = Invoice.InvoiceState.UNPAID
self.end_date = date
self.tax = tax_percentage * self.subtotal / 100
self.total = self.tax + self.subtotal
self.save()
def finish(self):
self.state = Invoice.InvoiceState.FINISHED
self.finish_date = timezone.now()
self.save()
def rollback_to_unpaid(self):
self.state = Invoice.InvoiceState.UNPAID
self.finish_date = None
self.save()
# end region
# region Invoice Component
class InvoiceInstance(BaseModel, InvoiceComponentMixin):
invoice = models.ForeignKey('Invoice', on_delete=models.CASCADE, related_name=labels.LABEL_INSTANCES)
# Key
instance_id = models.CharField(max_length=266)
# Price Dependency
flavor_id = models.CharField(max_length=256)
# Informative
name = models.CharField(max_length=256)
class InvoiceFloatingIp(BaseModel, InvoiceComponentMixin):
invoice = models.ForeignKey('Invoice', on_delete=models.CASCADE, related_name=labels.LABEL_FLOATING_IPS)
# Key
fip_id = models.CharField(max_length=266)
# Informative
ip = models.CharField(max_length=256)
class InvoiceVolume(BaseModel, InvoiceComponentMixin):
invoice = models.ForeignKey('Invoice', on_delete=models.CASCADE, related_name=labels.LABEL_VOLUMES)
# Key
volume_id = models.CharField(max_length=256)
# Price Dependency
volume_type_id = models.CharField(max_length=256)
space_allocation_gb = models.FloatField()
# Informative
volume_name = models.CharField(max_length=256)
@property
def price_charged(self):
price_without_allocation = super().price_charged
return price_without_allocation * math.ceil(self.space_allocation_gb)
class InvoiceRouter(BaseModel, InvoiceComponentMixin):
invoice = models.ForeignKey('Invoice', on_delete=models.CASCADE, related_name=labels.LABEL_ROUTERS)
# Key
router_id = models.CharField(max_length=256)
# Informative
name = models.CharField(max_length=256)
class InvoiceSnapshot(BaseModel, InvoiceComponentMixin):
invoice = models.ForeignKey('Invoice', on_delete=models.CASCADE, related_name=labels.LABEL_SNAPSHOTS)
# Key
snapshot_id = models.CharField(max_length=256)
# Price Dependency
space_allocation_gb = models.FloatField()
# Informative
name = models.CharField(max_length=256)
@property
def price_charged(self):
price_without_allocation = super().price_charged
return price_without_allocation * math.ceil(self.space_allocation_gb)
class InvoiceImage(BaseModel, InvoiceComponentMixin):
invoice = models.ForeignKey('Invoice', on_delete=models.CASCADE, related_name=labels.LABEL_IMAGES)
# Key
image_id = models.CharField(max_length=256)
# Price Dependency
space_allocation_gb = models.FloatField()
# Informative
name = models.CharField(max_length=256)
@property
def price_charged(self):
price_without_allocation = super().price_charged
return price_without_allocation * math.ceil(self.space_allocation_gb)
# end region

0
core/utils/__init__.py Normal file
View file

View file

@ -0,0 +1,63 @@
import json
from core.models import DynamicSetting
BILLING_ENABLED = "billing_enabled"
INVOICE_TAX = "invoice_tax"
DEFAULTS = {
BILLING_ENABLED: False,
INVOICE_TAX: 11
}
def _get_casted_value(setting: DynamicSetting):
if setting.type == DynamicSetting.DataType.JSON:
return json.loads(setting.value)
elif setting.type == DynamicSetting.DataType.BOOLEAN:
return setting.value == "1"
elif setting.type == DynamicSetting.DataType.INT:
return int(setting.value)
elif setting.type == DynamicSetting.DataType.STR:
return setting.value
else:
raise ValueError("Type not supported")
def get_dynamic_settings():
result = DEFAULTS.copy()
settings = DynamicSetting.objects.all()
for setting in settings:
result[setting.key] = _get_casted_value(setting)
return result
def get_dynamic_setting(key):
setting: DynamicSetting = DynamicSetting.objects.filter(key=key).first()
if not setting:
return DEFAULTS[key]
return setting.value
def set_dynamic_setting(key, value):
if type(value) is dict:
inserted_value = json.dumps(value)
data_type = DynamicSetting.DataType.JSON
elif type(value) is bool:
inserted_value = "1" if value else "0"
data_type = DynamicSetting.DataType.BOOLEAN
elif type(value) is int:
inserted_value = str(value)
data_type = DynamicSetting.DataType.INT
elif type(value) is str:
inserted_value = value
data_type = DynamicSetting.DataType.STR
else:
raise ValueError("Type not supported")
DynamicSetting.objects.update_or_create(key=key, defaults={
"value": inserted_value,
"type": data_type
})

88
core/utils/model_utils.py Normal file
View file

@ -0,0 +1,88 @@
import math
from django.db import models
from django.utils import timezone
from djmoney.models.fields import MoneyField
class BaseModel(models.Model):
class Meta:
abstract = True
class TimestampMixin(models.Model):
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class Meta:
abstract = True
class PriceMixin(models.Model):
hourly_price = MoneyField(max_digits=10, decimal_places=0)
monthly_price = MoneyField(max_digits=10, decimal_places=0, default=None, blank=True, null=True)
class Meta:
abstract = True
class InvoiceComponentMixin(TimestampMixin, PriceMixin):
"""
Storing start time for price calculation
"""
start_date = models.DateTimeField()
"""
Storing end time of the component, when component still active it will be None.
It will be set when component is closed or rolled
"""
end_date = models.DateTimeField(default=None, blank=True, null=True)
@property
def adjusted_end_date(self):
"""
Get end date that will be used for calculation. Please use this for calculation and displaying usage duration
Basically it just return current time if end_date is None
end_date will be set when invoice is finished every end of the month or when invoice component is rolled
"""
current_date = timezone.now()
if self.end_date:
end_date = self.end_date
else:
end_date = current_date
return end_date
@property
def price_charged(self):
"""
Calculate the price to be charged to user
"""
end_date = self.adjusted_end_date
if self.start_date.date().day == 1 and end_date.date().day == 1 \
and self.start_date.date().month != end_date.date().month \
and self.monthly_price:
# Using monthly price
return self.monthly_price
seconds_passes = (end_date - self.start_date).total_seconds()
hour_passes = math.ceil(seconds_passes / 3600)
return self.hourly_price * hour_passes
def is_closed(self):
"""
Is component closed
"""
return self.end_date is not None
def close(self, date):
"""
Close component the component
"""
self.end_date = date
self.save()
class Meta:
abstract = True

22
manage.py Executable file
View file

@ -0,0 +1,22 @@
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys
def main():
"""Run administrative tasks."""
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'yuyu.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
if __name__ == '__main__':
main()

11
requirements.txt Normal file
View file

@ -0,0 +1,11 @@
asgiref==3.4.1
Django==3.2.6
pytz==2021.1
sqlparse==0.4.1
oslo.messaging>=12.8.0
django-money==2.0.1
openstacksdk==0.58.0
djangorestframework==3.12.4
django-filter==2.4.0
Markdown==3.3.4
gunicorn==20.1.0

12
script/yuyu_api.service Normal file
View file

@ -0,0 +1,12 @@
[Unit]
Description=yuyu api daemon
After=network.target
[Service]
User=root
Group=root
WorkingDirectory={{yuyu_dir}}
ExecStart={{yuyu_dir}}/env/bin/gunicorn yuyu.wsgi --workers 2 --bind 127.0.0.1:8182 --log-file=logs/gunicorn.log
[Install]
WantedBy=multi-user.target

View file

@ -0,0 +1,12 @@
[Unit]
Description=yuyu event monitor daemon
After=network.target
[Service]
User=root
Group=root
WorkingDirectory={{yuyu_dir}}
ExecStart={{yuyu_dir}}/env/bin/python manage.py event_monitor
[Install]
WantedBy=multi-user.target

0
yuyu/__init__.py Normal file
View file

16
yuyu/asgi.py Normal file
View file

@ -0,0 +1,16 @@
"""
ASGI config for yuyu project.
It exposes the ASGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/3.2/howto/deployment/asgi/
"""
import os
from django.core.asgi import get_asgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'yuyu.settings')
application = get_asgi_application()

View file

@ -0,0 +1,7 @@
YUYU_NOTIFICATION_URL = "rabbit://openstack:password@127.0.0.1:5672/"
YUYU_NOTIFICATION_TOPICS = ["notifications"]
# Currency Configuration
CURRENCIES = ('IDR', 'USD')
DEFAULT_CURRENCY = "IDR"

150
yuyu/settings.py Normal file
View file

@ -0,0 +1,150 @@
"""
Django settings for yuyu project.
Generated by 'django-admin startproject' using Django 3.2.6.
For more information on this file, see
https://docs.djangoproject.com/en/3.2/topics/settings/
For the full list of settings and their values, see
https://docs.djangoproject.com/en/3.2/ref/settings/
"""
from pathlib import Path
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/3.2/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-insecure-kt=8eo4p*(5uq4jj^=)l8_4-_&44(#9xp!m0+^+csdok=(@ug3'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
ALLOWED_HOSTS = []
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'djmoney',
'rest_framework',
'core',
'api',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'yuyu.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [BASE_DIR / 'templates']
,
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'yuyu.wsgi.application'
# Database
# https://docs.djangoproject.com/en/3.2/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
}
}
# Password validation
# https://docs.djangoproject.com/en/3.2/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
# Internationalization
# https://docs.djangoproject.com/en/3.2/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.2/howto/static-files/
STATIC_URL = '/static/'
# Default primary key field type
# https://docs.djangoproject.com/en/3.2/ref/settings/#default-auto-field
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
# Logging
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'class': 'logging.StreamHandler',
},
},
'root': {
'handlers': ['console'],
'level': 'INFO',
},
}
try:
from .local_settings import *
except ImportError:
pass

22
yuyu/urls.py Normal file
View file

@ -0,0 +1,22 @@
"""yuyu URL Configuration
The `urlpatterns` list routes URLs to views. For more information please see:
https://docs.djangoproject.com/en/3.2/topics/http/urls/
Examples:
Function views
1. Add an import: from my_app import views
2. Add a URL to urlpatterns: path('', views.home, name='home')
Class-based views
1. Add an import: from other_app.views import Home
2. Add a URL to urlpatterns: path('', Home.as_view(), name='home')
Including another URLconf
1. Import the include() function: from django.urls import include, path
2. Add a URL to urlpatterns: path('blog/', include('blog.urls'))
"""
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('api/', include('api.urls'))
]

16
yuyu/wsgi.py Normal file
View file

@ -0,0 +1,16 @@
"""
WSGI config for yuyu project.
It exposes the WSGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/3.2/howto/deployment/wsgi/
"""
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'yuyu.settings')
application = get_wsgi_application()